-- Logs begin at Sun 2022-09-11 00:10:37 UTC, end at Sun 2022-09-11 01:13:00 UTC. -- Sep 11 00:40:57.738431 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Sep 11 00:40:57.745966 dv-cinder-master-3-smb devstack@keystone.service[81474]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 11 00:40:57 2022] *** Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: nodename: dv-cinder-master-3-smb Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: machine: x86_64 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: clock source: unix Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: pcre jit disabled Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: detected number of CPU cores: 4 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: current working directory: / Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: detected binary path: /usr/bin/uwsgi-core Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: your processes number limit is 47680 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: your memory page size is 4096 bytes Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: detected max file descriptor number: 2048 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: lock engine: pthread robust mutexes Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: thunder lock: enabled Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 11 00:40:57.750008 dv-cinder-master-3-smb devstack@keystone.service[81474]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 11 00:40:57.787693 dv-cinder-master-3-smb devstack@keystone.service[81474]: Python main interpreter initialized at 0x55c91730c9c0 Sep 11 00:40:57.787693 dv-cinder-master-3-smb devstack@keystone.service[81474]: python threads support enabled Sep 11 00:40:57.787693 dv-cinder-master-3-smb devstack@keystone.service[81474]: your server socket listen backlog is limited to 100 connections Sep 11 00:40:57.787693 dv-cinder-master-3-smb devstack@keystone.service[81474]: your mercy for graceful operations on workers is 90 seconds Sep 11 00:40:57.788064 dv-cinder-master-3-smb devstack@keystone.service[81474]: mapped 403077 bytes (393 KB) for 2 cores Sep 11 00:40:57.788145 dv-cinder-master-3-smb devstack@keystone.service[81474]: *** Operational MODE: preforking *** Sep 11 00:40:57.788400 dv-cinder-master-3-smb devstack@keystone.service[81474]: *** uWSGI is running in multiple interpreter mode *** Sep 11 00:40:57.788400 dv-cinder-master-3-smb devstack@keystone.service[81474]: spawned uWSGI master process (pid: 81474) Sep 11 00:40:57.788646 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Sep 11 00:40:57.791267 dv-cinder-master-3-smb devstack@keystone.service[81474]: spawned uWSGI worker 1 (pid: 81487, cores: 1) Sep 11 00:40:57.791267 dv-cinder-master-3-smb devstack@keystone.service[81474]: spawned uWSGI worker 2 (pid: 81488, cores: 1) Sep 11 00:40:57.791267 dv-cinder-master-3-smb devstack@keystone.service[81474]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 11 00:41:02.017444 dv-cinder-master-3-smb devstack@keystone.service[81488]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:41:02.017444 dv-cinder-master-3-smb devstack@keystone.service[81488]: from cryptography.utils import int_from_bytes Sep 11 00:41:02.018296 dv-cinder-master-3-smb devstack@keystone.service[81488]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:41:02.018296 dv-cinder-master-3-smb devstack@keystone.service[81488]: from cryptography.utils import int_from_bytes Sep 11 00:41:02.019058 dv-cinder-master-3-smb devstack@keystone.service[81487]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:41:02.019058 dv-cinder-master-3-smb devstack@keystone.service[81487]: from cryptography.utils import int_from_bytes Sep 11 00:41:02.019608 dv-cinder-master-3-smb devstack@keystone.service[81487]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 11 00:41:02.019608 dv-cinder-master-3-smb devstack@keystone.service[81487]: from cryptography.utils import int_from_bytes Sep 11 00:41:02.067218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 00:41:02.067432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 00:41:02.067606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] command line args: [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 00:41:02.067872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 00:41:02.068016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ================================================================================ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 00:41:02.068291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] admin_token = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.068538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] config_dir = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.068689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 11 00:41:02.068780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.068859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 11 00:41:02.068936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] config_source = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.068998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] command line args: [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 11 00:41:02.069073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] debug = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.069150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 11 00:41:02.069211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.070609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ================================================================================ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 11 00:41:02.070684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.070757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] admin_token = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.070817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.070881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] config_dir = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.070939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] config_source = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] debug = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_config_append = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_dir = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.071993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_config_append = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_dir = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.072904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.073965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] publish_errors = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.075995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] standard_threads = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] publish_errors = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.076952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] use_journal = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] use_json = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] use_stderr = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] use_syslog = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] standard_threads = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.077879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.078000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.078117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.078233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.078343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.078451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.078568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.078678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] use_journal = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.078678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] use_json = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.078896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.079005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] use_stderr = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.079120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.079275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] use_syslog = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.079395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.079511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 11 00:41:02.079621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.079729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.079844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.079961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.080998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.081998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.082993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.083996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.084917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085242 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.085904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.086927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.087953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.088883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.089925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.090923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.091861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.092906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.093978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.094933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095264 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.095933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.096930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.097933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.098932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.099937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.100955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.101974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.external = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.102988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.password = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] auth.token = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.103930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.external = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104647 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.104996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.password = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] auth.token = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105782 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.105892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.106897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.107963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.108903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.109936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.110956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.111990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.112902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.113922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.114995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.115909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116250 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.116940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117836 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.117947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.118908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.119948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120058 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.120958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.121969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.122977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123363 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123363 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.123916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.124917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.125927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.126927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.127967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.128974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129312 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.129991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.130998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.131919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.132925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.133901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] role.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] role.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134782 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.134894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.135999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] role.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.136894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] role.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137765 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.137884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.138975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.139915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140664 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.140956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.141999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.142953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.143962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.144834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.145950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.146932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.147905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.148928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.149943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81488) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 00:41:02.150063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.150063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.150281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.150394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.150635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.150871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.151115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.151369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.151608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.151840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 11 00:41:02.152008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81487) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 11 00:41:02.178714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.178941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.179162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.180075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.180308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.180531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.187833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.189215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.215801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.216041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.217146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.217377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.268238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.268795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.316137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.316367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.317819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.318052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.473825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.474991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.475128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.475333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.475536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.475751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.497496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.497702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.497912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498836 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.498986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.499170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.499433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.509822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.510515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.522486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.522617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.522855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81488) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.523923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.524128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.524342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81487) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 11 00:41:02.540254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.540459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 00:41:02.540805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.541000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 00:41:02.541266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.541405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.541488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 00:41:02.541563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.541769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.541894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.541978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 11 00:41:02.542056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.542242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.542505 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.542767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.543023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.551555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.552099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.552379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.552588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.552868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.553141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.553428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.559326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.560422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.562546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.562833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.563112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.563419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.563697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.563821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.563977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.564816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.565019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.565144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.565292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.565376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.565575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.565847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.566124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.566396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.587714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.588962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.589178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.589398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.590426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.590715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.594054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.594342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.594624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.595396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.595686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.595974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.600971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.601257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.602368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.602659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.606828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.607125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.607439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.607722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.607998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.608254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.608344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.608520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.608605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.608789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.608886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.609095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.609360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.609640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.609913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.610188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.624638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.624909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.626748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.627027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.628749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.630975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.632057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.632354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.634267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.634561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.636557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.638766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.639700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.641880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.642879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.643212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.643484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.643772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.644049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.644335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.645264 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.645566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.645853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.646138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.646442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.646711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.660052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.660341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.660663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.660925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.662540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.662830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.663101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.663394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.667747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.669283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.670218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.670795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.671070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.671784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.673306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.673584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.673725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.674039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.674323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.674606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.674998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.676301 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.676590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.676879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.677166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.677550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.686484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.686779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.687055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.687354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.687734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.688007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.688286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.689091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.689385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.689671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.689956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.690344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.690612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.690893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.703634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.706344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.706473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.709301 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.709506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.710993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.712217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.712467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.712751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.713146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.713665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.715123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.715434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.715807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.719480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.720929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.722170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.723742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.723865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.724243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.724528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.724805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.726630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.727024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.727337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.727620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.732601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.732892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.733180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.735425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.735730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.736007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.740057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81488) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.740342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.740622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.740899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.741175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.741445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.741717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.741997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.742265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.742544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.742825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.742949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81487) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 11 00:41:02.743147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.743313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.743447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.743564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.743685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.743801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81488) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.743975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.744248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.744536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.744819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.745092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.745372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.745643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.745920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.746187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.746442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81487) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 11 00:41:02.777378 dv-cinder-master-3-smb devstack@keystone.service[81488]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c91730c9c0 pid: 81488 (default app) Sep 11 00:41:02.780431 dv-cinder-master-3-smb devstack@keystone.service[81487]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c91730c9c0 pid: 81487 (default app) Sep 11 00:41:02.780556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d7c8-796c-4d7e-a9fc-f743e1f349a5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:02.780844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d7c8-796c-4d7e-a9fc-f743e1f349a5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:02.781113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0d7c8-796c-4d7e-a9fc-f743e1f349a5 None None] PATH_INFO: `/v3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:02.785285 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Sun Sep 11 00:41:02 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:09.672689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:09.673567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:09.673567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:09.679007 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:09.684530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:09.685059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:09.685059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:09.754114 dv-cinder-master-3-smb devstack@keystone.service[81488]: INFO dbcounter [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] Registered counter for database keystone Sep 11 00:41:09.775147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG oslo_db.sqlalchemy.engines [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81488) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:41:09.783057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writer thread running {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:41:09.971171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG passlib.handlers.bcrypt [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81488) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 00:41:09.971493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG passlib.handlers.bcrypt [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81488) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 00:41:10.030252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:10.031870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:10.164764 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 00:41:10.665413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:10.745054 dv-cinder-master-3-smb devstack@keystone.service[81487]: INFO dbcounter [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] Registered counter for database keystone Sep 11 00:41:10.769954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG oslo_db.sqlalchemy.engines [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81487) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 11 00:41:10.781641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writer thread running {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 11 00:41:10.902476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0df88-1b28-412b-91d8-940af93e9a02 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:10.908097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:10.908721 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:10.908990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:10.909257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:10.917942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:10.969582 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING keystone.common.rbac_enforcer.enforcer [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 00:41:10.969925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af968963-2a51-4f3b-ac3b-432c1f5efce8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:10.980154 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Sun Sep 11 00:41:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:41:10.989414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:11.013491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158bbbe-3130-40df-a7e8-1fb81c364e6b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:11.019309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:11.020456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:11.020727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:11.020991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] PATH_INFO: `/v3/projects/857538f273814e30bf74c0248d309754` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:11.022733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] RBAC: Authorizing `identity:get_project(project_id=857538f273814e30bf74c0248d309754)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:11.075381 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING keystone.common.rbac_enforcer.enforcer [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 11 00:41:11.075716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f59d6-da65-4492-a67a-42018661d8df admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:11.080246 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Sep 11 00:41:10 2022] GET /identity/v3/projects/857538f273814e30bf74c0248d309754 => generated 344 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:13.203539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a87eb-742a-41e5-88ea-1b5b1868f099 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:13.204027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a87eb-742a-41e5-88ea-1b5b1868f099 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:13.204144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a87eb-742a-41e5-88ea-1b5b1868f099 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:13.210729 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:13.221119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:13.222015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:13.222015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:13.243633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:13.245141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:13.254531 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:13 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 11 00:41:13.527011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a87eb-742a-41e5-88ea-1b5b1868f099 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:13.541469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a87eb-742a-41e5-88ea-1b5b1868f099 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:13.546963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55daba21-2ea1-49ab-83c6-66a3381482c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:13.547585 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daba21-2ea1-49ab-83c6-66a3381482c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:13.547857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daba21-2ea1-49ab-83c6-66a3381482c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:13.548124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55daba21-2ea1-49ab-83c6-66a3381482c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:13.556219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daba21-2ea1-49ab-83c6-66a3381482c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:13.557969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55daba21-2ea1-49ab-83c6-66a3381482c2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:13.567300 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Sun Sep 11 00:41:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:41:13.573364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:13.585754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722f94b-3854-4207-9054-18f1f6b38137 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:13.591638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2516a5a-650b-4efc-b033-44ac887a2b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:13.592274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2516a5a-650b-4efc-b033-44ac887a2b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:13.592542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2516a5a-650b-4efc-b033-44ac887a2b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:13.592810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2516a5a-650b-4efc-b033-44ac887a2b58 admin admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:13.594284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2516a5a-650b-4efc-b033-44ac887a2b58 admin admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:13.596216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2516a5a-650b-4efc-b033-44ac887a2b58 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:13.597750 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Sun Sep 11 00:41:13 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:15.798384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fac1a-6403-4742-83b2-99ddcb53402d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:15.798958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fac1a-6403-4742-83b2-99ddcb53402d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:15.798958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7fac1a-6403-4742-83b2-99ddcb53402d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:15.800262 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:15.805686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:15.806142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:15.806238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:15.825627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:15.826804 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:15.840748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e6d0-ff1d-4361-8d36-b3c89a2c09ba None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:15.841152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e6d0-ff1d-4361-8d36-b3c89a2c09ba None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:15.841421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e6d0-ff1d-4361-8d36-b3c89a2c09ba None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:15.842704 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:41:15.847724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:15.848005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:15.848277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:15.889824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG passlib.handlers.bcrypt [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81487) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 11 00:41:15.890197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG passlib.handlers.bcrypt [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81487) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 11 00:41:15.910489 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:15.938991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:15.940629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:16.040636 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:16.175687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.200756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.282743 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e4bd-57b4-49a2-ab6e-849b8ec81f28 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.299328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.319598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.320243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.320521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.320789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.331835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:16.340726 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:16.341793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:16.342647 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:16.347887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.400958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c7477-2cd3-4b85-af85-eb59fb3b1b3f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.403210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.428632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.437446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.438316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.438593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.438859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.446903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012b084-93ce-489d-8645-2c1ced175fc5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.479669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.480274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.480550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.480817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.481485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:16.481787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.483208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.494700 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:16.500617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.504183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.505695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.603125 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:41:16.609120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.610195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a2ada4-71e8-4f57-85be-8d0557aa6732 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.612015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.644268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.645037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.645315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.645586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.646736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.648228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.649839 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:16.655528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.670606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.710746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835549e-9b7f-408c-bcf2-73c3ae7e625c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.745938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.746801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.747076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.747373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.754573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d84b00-96fe-4d30-931b-5d394e3b2a44 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.765999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.767843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:41:16.786439 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:16.787396 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:41:16.788207 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:41:16.788207 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:41:16.788207 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:16.788207 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:16.791884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.792525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.792819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.793091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.793549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.794280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.795908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.797454 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:16.803536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:16.852166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.885369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:16.900762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e5af7-6cfe-460c-bdfd-3b864ed914a7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.905958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44a99d-8a07-4f49-a681-0ba000a99958 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:16.936362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.936964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.937237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.937508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.938215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:16.938487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.939933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d9639-d721-4c79-ae39-3dec6319e6b8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.940094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:16.940695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:16.940986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:16.941260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:16.942020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.domain.id=default, user.id=cf7ec619bcef4004bcdaf4a26a07260b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:16.942324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:16.943827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b39b6a1-34c1-4e40-b5ef-77af3fb66801 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:16.950616 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/role_assignments?user.id=cf7ec619bcef4004bcdaf4a26a07260b&scope.domain.id=default&role.id=77132358e0b64de4a63750b81e359340 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:16.958021 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:19.222518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292fdb8-db87-4aeb-80a0-9195ee45352f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:19.223220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292fdb8-db87-4aeb-80a0-9195ee45352f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:19.223220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292fdb8-db87-4aeb-80a0-9195ee45352f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:19.224833 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:19.230820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb57f44-97d4-425c-9f40-958d67b84dd9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:19.231691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb57f44-97d4-425c-9f40-958d67b84dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:19.232161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb57f44-97d4-425c-9f40-958d67b84dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:19.256193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6bb57f44-97d4-425c-9f40-958d67b84dd9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:19.257805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6bb57f44-97d4-425c-9f40-958d67b84dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:19.348884 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:19.817466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c84a-1314-484a-85d5-5bf7633c6847 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:19.818180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c84a-1314-484a-85d5-5bf7633c6847 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:19.818652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c84a-1314-484a-85d5-5bf7633c6847 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:19.820247 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 13/24] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:19.837329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a068a8-77ac-43db-85ad-732c02e1b394 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:19.837604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a068a8-77ac-43db-85ad-732c02e1b394 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:19.838079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a068a8-77ac-43db-85ad-732c02e1b394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:19.859850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-81a068a8-77ac-43db-85ad-732c02e1b394 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:19.861330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-81a068a8-77ac-43db-85ad-732c02e1b394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:19.892800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407c84a-1314-484a-85d5-5bf7633c6847 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:19.929829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:19.980181 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 12/25] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:19.981458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3149d5c-927b-4f6d-b00d-9b99c6654a14 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:19.981738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3149d5c-927b-4f6d-b00d-9b99c6654a14 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:19.982008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3149d5c-927b-4f6d-b00d-9b99c6654a14 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:19.983254 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 13/26] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:19.988979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f81dc-9d3f-490c-9e3e-e1163c93d8bf None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:19.989252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f81dc-9d3f-490c-9e3e-e1163c93d8bf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:19.989523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f81dc-9d3f-490c-9e3e-e1163c93d8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.024302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-998f81dc-9d3f-490c-9e3e-e1163c93d8bf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:20.025543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-998f81dc-9d3f-490c-9e3e-e1163c93d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:20.074314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407c84a-1314-484a-85d5-5bf7633c6847 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:20.109891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4da189-43b1-4abf-8702-ab84e75e6dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:20.110894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da189-43b1-4abf-8702-ab84e75e6dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.111331 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da189-43b1-4abf-8702-ab84e75e6dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.111832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4da189-43b1-4abf-8702-ab84e75e6dc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.124048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4da189-43b1-4abf-8702-ab84e75e6dc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-dd4da189-43b1-4abf-8702-ab84e75e6dc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:20.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:20.131130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:20.132121 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:20.134375 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 14/27] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:20.136064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ee1d2-46ea-4857-a88d-7903c8d47e98 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.136607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ee1d2-46ea-4857-a88d-7903c8d47e98 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.137025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ee1d2-46ea-4857-a88d-7903c8d47e98 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.138421 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 15/28] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:20.146448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ee1d2-46ea-4857-a88d-7903c8d47e98 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:20.175810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:20.197970 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 14/29] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:20.199690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.199976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.200245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.236934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:20.238159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:20.329561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ee1d2-46ea-4857-a88d-7903c8d47e98 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:20.366722 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 15/30] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:20.385902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:20.386526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.386803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.387072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.387785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:20.388085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:20.389497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:20.415514 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 16/31] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:20.421196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:20.457391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:20.468837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:20.577652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c897c-60a7-4d7d-bd82-02ccbcdaa063 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:20.598163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:20.614733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:20.615623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.616025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.616447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.618052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:20.619796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:20.621880 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 16/32] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:20 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:20.622823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:20.661450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529276eb-48ae-471c-8b2d-915bec6e8066 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:20.721401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:20.722429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.722961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.723464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.734937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:20.736776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:20.738644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4567a-204a-4503-9260-28826d5113a9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:20.743166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:41:20.757013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:20.757945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:41:20.758775 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:41:20.758775 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:41:20.758775 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:20.761013 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 17/33] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:20.761933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:20.778224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:20.778929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.779410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.779684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.780359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:20.781728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:20.879353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a2314-aebe-41cb-88d3-12712bb78f45 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:20.897444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:20.917869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:20.918627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:20.918967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:20.919331 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:20.921672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:20.923370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:20.925139 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 18/34] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:20.926045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:20.926847 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 17/35] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Sep 11 00:41:20 2022] POST /identity/v3/roles => generated 219 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:20.928156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:21.098056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108ad9c-368a-4c56-97f1-c8402caec2ca None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:21.103141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1d7fc-f5e6-4cca-8dc7-03adf8e378ce None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:21.146259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:21.146811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.147095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.147398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.147982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5fa35-74ad-4b9d-a171-57e8b79e5b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:21.148095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:21.148367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:21.149059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5fa35-74ad-4b9d-a171-57e8b79e5b77 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.149502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5fa35-74ad-4b9d-a171-57e8b79e5b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.149894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:21.150107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5fa35-74ad-4b9d-a171-57e8b79e5b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.150952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5fa35-74ad-4b9d-a171-57e8b79e5b77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:21.152824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5fa35-74ad-4b9d-a171-57e8b79e5b77 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:21.167340 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 18/36] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:21.168217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:21.206020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:21.289389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4299dfd-80c3-4943-b308-5b094eb2cdc7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:21.290010 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 19/37] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Sep 11 00:41:20 2022] POST /identity/v3/roles => generated 225 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:21.290929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0ab0c-f7bb-497e-accc-126a9a89698d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.291308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0ab0c-f7bb-497e-accc-126a9a89698d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.291507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0ab0c-f7bb-497e-accc-126a9a89698d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.292834 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 20/38] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:21.297425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0ab0c-f7bb-497e-accc-126a9a89698d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:21.337644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98348c29-e84e-426f-b25b-b7df86e9b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:21.338349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348c29-e84e-426f-b25b-b7df86e9b2a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.338628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348c29-e84e-426f-b25b-b7df86e9b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.338898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98348c29-e84e-426f-b25b-b7df86e9b2a9 None admin] PATH_INFO: `/v3/domains/default/users/cf7ec619bcef4004bcdaf4a26a07260b/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.341622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98348c29-e84e-426f-b25b-b7df86e9b2a9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cf7ec619bcef4004bcdaf4a26a07260b, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:21.343101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98348c29-e84e-426f-b25b-b7df86e9b2a9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:21.348581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:21.364567 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 19/39] 10.222.0.23 () {58 vars in 1361 bytes} [Sun Sep 11 00:41:21 2022] PUT /identity/v3/domains/default/users/cf7ec619bcef4004bcdaf4a26a07260b/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:21.365647 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15643a-bdb0-40b8-8456-9c78da3b1ef0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.365932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15643a-bdb0-40b8-8456-9c78da3b1ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.366210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15643a-bdb0-40b8-8456-9c78da3b1ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.387355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4e15643a-bdb0-40b8-8456-9c78da3b1ef0 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:21.388167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4e15643a-bdb0-40b8-8456-9c78da3b1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:21.443030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0ab0c-f7bb-497e-accc-126a9a89698d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:21.475212 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 20/40] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:21.475212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2def02-a1db-454f-8ff7-d68e996d8aa0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.475212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2def02-a1db-454f-8ff7-d68e996d8aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.475212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2def02-a1db-454f-8ff7-d68e996d8aa0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.475212 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 21/41] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:21.478655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafb653-fb49-4681-881b-b702206af680 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.478946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafb653-fb49-4681-881b-b702206af680 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.479238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafb653-fb49-4681-881b-b702206af680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.502590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9cafb653-fb49-4681-881b-b702206af680 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:21.503834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9cafb653-fb49-4681-881b-b702206af680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:21.506380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33260176-5164-45d3-8364-7656ea62050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:21.507723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33260176-5164-45d3-8364-7656ea62050c None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.507723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33260176-5164-45d3-8364-7656ea62050c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.507723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33260176-5164-45d3-8364-7656ea62050c None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.508268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33260176-5164-45d3-8364-7656ea62050c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:21.510073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33260176-5164-45d3-8364-7656ea62050c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:21.516595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:21.554220 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 21/42] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:41:21 2022] POST /identity/v3/projects => generated 301 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:21.593868 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 22/43] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:21.784646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33260176-5164-45d3-8364-7656ea62050c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:21.876720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafb653-fb49-4681-881b-b702206af680 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:21.886682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:21.891438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33260176-5164-45d3-8364-7656ea62050c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:21.929740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:21.930508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:21.930912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:21.931308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:21.932117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:21.933512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:21.953295 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Sep 11 00:41:21 2022] POST /identity/v3/roles => generated 223 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:21.987011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafb653-fb49-4681-881b-b702206af680 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:22.026879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239deb5-a22f-492c-85db-2deb88d55b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:22.027821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239deb5-a22f-492c-85db-2deb88d55b78 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:22.028283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239deb5-a22f-492c-85db-2deb88d55b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:22.028691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239deb5-a22f-492c-85db-2deb88d55b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:22.030088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239deb5-a22f-492c-85db-2deb88d55b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:22.031759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239deb5-a22f-492c-85db-2deb88d55b78 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:22.033720 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:22.039851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:22.271093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dcd5f-5fd0-4279-9c4f-1f0920f451fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:22.306301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7f6ce-dd1e-4aa8-ae65-151a76430aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:22.306849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7f6ce-dd1e-4aa8-ae65-151a76430aec None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:22.307118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7f6ce-dd1e-4aa8-ae65-151a76430aec None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:22.307416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7f6ce-dd1e-4aa8-ae65-151a76430aec None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:22.308076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7f6ce-dd1e-4aa8-ae65-151a76430aec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:22.309630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7f6ce-dd1e-4aa8-ae65-151a76430aec None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:22.336588 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:41:22 2022] POST /identity/v3/projects => generated 312 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:23.624695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34a6c-c8a2-4fdc-a9f4-ce4eca8d6497 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:23.625243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34a6c-c8a2-4fdc-a9f4-ce4eca8d6497 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:23.625243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff34a6c-c8a2-4fdc-a9f4-ce4eca8d6497 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:23.626559 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:23.632127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:23.632529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:23.632705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:23.651906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:23.653406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:23.656165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:23.725380 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:23.984306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34a6c-c8a2-4fdc-a9f4-ce4eca8d6497 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:24.000176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:24.088441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff34a6c-c8a2-4fdc-a9f4-ce4eca8d6497 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:24.121917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:24.122545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.122816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.123051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.134296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:24.141475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:24.142348 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:24.143101 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:24.148597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:24.248580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5860c1b2-5e99-4870-a713-b2da6ef4f02e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:24.282322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:24.282928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.283206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.283496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.284154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:24.284453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:24.285892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7707454-0c4b-468d-96dc-153cf0636b77 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:24.294104 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 25/50] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:24.300059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:24.465190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a949a0-b776-462b-9f39-9b01dcecc4ea None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.465488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a949a0-b776-462b-9f39-9b01dcecc4ea None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.465710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a949a0-b776-462b-9f39-9b01dcecc4ea None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.467058 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 26/51] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:24.472086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.472367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.472639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.479230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:24.482981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:24.492861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:24.494093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:24.513901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953327e-d285-4aeb-b00f-a2a7a7902d9f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:24.548210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:24.549281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.549554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.549824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.551028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:24.552539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:24.554115 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 26/52] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:24 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:24.559526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:24.562800 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 27/53] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:24.656330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992aa64a-d9ac-4d3d-af97-7e46b4974c1b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:24.690116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:24.690733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.691004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.691282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.692438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:24.693869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:24.695445 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 27/54] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:24.701502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:24.807420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa973719-c420-43cb-af24-42b1489931e7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:24.823900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:24.835398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:24.837843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:24.843318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:24.843932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.844210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.844479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.845202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.domain.id=default, user.id=cf7ec619bcef4004bcdaf4a26a07260b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:24.845508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:24.846975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:24.853211 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1322 bytes} [Sun Sep 11 00:41:24 2022] GET /identity/v3/role_assignments?user.id=cf7ec619bcef4004bcdaf4a26a07260b&scope.domain.id=default&role.id=77132358e0b64de4a63750b81e359340 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:24.932113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae2246-12cf-4ea6-9144-01ff15d295c8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:24.969642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ba25b-03cf-476f-bc3f-25b509e37a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:24.970280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ba25b-03cf-476f-bc3f-25b509e37a21 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:24.970556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ba25b-03cf-476f-bc3f-25b509e37a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:24.970825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ba25b-03cf-476f-bc3f-25b509e37a21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:24.972018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ba25b-03cf-476f-bc3f-25b509e37a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:24.973612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ba25b-03cf-476f-bc3f-25b509e37a21 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:24.975444 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:24.981273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:25.090375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d431d35-2a42-4d5c-846a-d847f30cde0f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:25.123053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b5125-0ee0-46ac-8961-b8667521c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:25.123712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b5125-0ee0-46ac-8961-b8667521c00f None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:25.123979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b5125-0ee0-46ac-8961-b8667521c00f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:25.124245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b5125-0ee0-46ac-8961-b8667521c00f None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:25.124885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b5125-0ee0-46ac-8961-b8667521c00f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:25.126337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b5125-0ee0-46ac-8961-b8667521c00f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:25.153769 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 29/57] 10.222.0.23 () {60 vars in 1140 bytes} [Sun Sep 11 00:41:24 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:27.299717 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd160b90-41c6-4ec1-9974-566912eab1a7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:27.300539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd160b90-41c6-4ec1-9974-566912eab1a7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:27.300689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd160b90-41c6-4ec1-9974-566912eab1a7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:27.302989 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:27.311462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e746b-bace-43eb-af77-546908fa2472 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:27.311893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e746b-bace-43eb-af77-546908fa2472 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:27.311977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e746b-bace-43eb-af77-546908fa2472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:27.330894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-725e746b-bace-43eb-af77-546908fa2472 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:27.332154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-725e746b-bace-43eb-af77-546908fa2472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:27.378464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:27.408642 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:27.676843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd160b90-41c6-4ec1-9974-566912eab1a7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:27.762303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:27.789164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd160b90-41c6-4ec1-9974-566912eab1a7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:27.825316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a34f45-e0fe-4bd3-9f52-70db455d1526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:27.825929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a34f45-e0fe-4bd3-9f52-70db455d1526 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:27.826212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a34f45-e0fe-4bd3-9f52-70db455d1526 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:27.826487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a34f45-e0fe-4bd3-9f52-70db455d1526 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:27.827765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a34f45-e0fe-4bd3-9f52-70db455d1526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:27.829233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a34f45-e0fe-4bd3-9f52-70db455d1526 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:27.830824 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:27.837371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e746b-bace-43eb-af77-546908fa2472 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:27.953062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e746b-bace-43eb-af77-546908fa2472 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:27.987776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ac1123-21ac-4643-8508-3eea4706edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:27.988394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1123-21ac-4643-8508-3eea4706edd5 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:27.988663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1123-21ac-4643-8508-3eea4706edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:27.988924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ac1123-21ac-4643-8508-3eea4706edd5 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:27.989568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac1123-21ac-4643-8508-3eea4706edd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:27.990999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ac1123-21ac-4643-8508-3eea4706edd5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:28.019884 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:41:27 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:31.244589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5ca21-e4c6-4a4d-8dc2-d49aae8b91d0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.245550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5ca21-e4c6-4a4d-8dc2-d49aae8b91d0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.246043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5ca21-e4c6-4a4d-8dc2-d49aae8b91d0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.247618 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:31.253218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadca3e-ee32-4493-91c4-2c00da2d828e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.254172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadca3e-ee32-4493-91c4-2c00da2d828e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.254633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadca3e-ee32-4493-91c4-2c00da2d828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.281750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3cadca3e-ee32-4493-91c4-2c00da2d828e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:31.283310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3cadca3e-ee32-4493-91c4-2c00da2d828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:31.323510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:31.379111 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 32/63] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:31.503978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c270ea2-f3e2-4c42-8631-8189703fe2a7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.504599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c270ea2-f3e2-4c42-8631-8189703fe2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.505068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c270ea2-f3e2-4c42-8631-8189703fe2a7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.506643 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 32/64] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:31.511896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bf361-19d6-4831-a1a8-407150b4f508 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.512179 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bf361-19d6-4831-a1a8-407150b4f508 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.512530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0bf361-19d6-4831-a1a8-407150b4f508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.540243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5f0bf361-19d6-4831-a1a8-407150b4f508 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:31.541446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f0bf361-19d6-4831-a1a8-407150b4f508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:31.629742 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 33/65] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:31.771052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec27b-054f-4b5b-8bc7-082475ac4259 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.771703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec27b-054f-4b5b-8bc7-082475ac4259 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.772215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedec27b-054f-4b5b-8bc7-082475ac4259 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.773994 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 33/66] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:31.779998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aacbf-3166-404d-983a-b744371aaf4d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.780445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aacbf-3166-404d-983a-b744371aaf4d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.780846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42aacbf-3166-404d-983a-b744371aaf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.797314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec27b-054f-4b5b-8bc7-082475ac4259 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:31.804911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f42aacbf-3166-404d-983a-b744371aaf4d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:31.806358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f42aacbf-3166-404d-983a-b744371aaf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:31.891839 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 34/67] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:31.894302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80444ec1-74ef-4809-ada2-988752d77cbf None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.894939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80444ec1-74ef-4809-ada2-988752d77cbf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.895421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80444ec1-74ef-4809-ada2-988752d77cbf None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.897122 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 35/68] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:31.904516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba34df-e56a-43d3-825b-8a34a96cf451 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:31.904856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba34df-e56a-43d3-825b-8a34a96cf451 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:31.905432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba34df-e56a-43d3-825b-8a34a96cf451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:31.940995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eeba34df-e56a-43d3-825b-8a34a96cf451 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:31.942284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eeba34df-e56a-43d3-825b-8a34a96cf451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:31.961923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedec27b-054f-4b5b-8bc7-082475ac4259 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:31.973610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.004163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.005187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.005737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.006013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.017527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:32.022518 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:32.023385 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:32.024163 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 34/69] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:32.024806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.029571 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 36/70] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:32.031246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.035861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2339f3-d9bd-429d-99f6-8da53b5df98f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.036418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2339f3-d9bd-429d-99f6-8da53b5df98f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.037748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2339f3-d9bd-429d-99f6-8da53b5df98f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.039676 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 37/71] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:32.040665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2339f3-d9bd-429d-99f6-8da53b5df98f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.155986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2339f3-d9bd-429d-99f6-8da53b5df98f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.166227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c401bbbd-94ba-40b1-a287-1b8e5408d70c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.191525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.192135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.192408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.192677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.193339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:32.193641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.195016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e838cc-8813-4703-b6a1-d608a56e243d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.203255 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 38/72] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:32.203399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.204032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.204389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.204662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.205070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.205461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.205858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.219493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:32.223889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:32.224328 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:32.225178 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:32.227974 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:32.228564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:32.228634 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 35/73] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:32.228634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.237374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.295813 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 39/74] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:32.297198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.300176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.337814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44f7a7-b95c-4581-bb98-d58a60dbc729 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.374002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.374628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.375012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.375325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.386529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:32.391795 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:32.392647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:32.393433 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 36/75] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:32.396364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.402887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3deef1-9f97-401d-9f79-7946deaf82a6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.439347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.439984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.440258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.440528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.453958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.455466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.457160 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 40/76] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:32.462559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.465933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.508315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc53f6-5a82-493c-9d21-9cded6c0b35c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.535531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.553681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.554281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.554561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.554834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.555535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:32.555838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.557230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.570086 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 37/77] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:32.570912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.571990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd061eb-d1bb-4309-a075-5a1187f8f92c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.609501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.610122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.610397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.610666 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.611842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.615066 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 41/78] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:32.615652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.675045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11a310-c3f6-4771-9e4d-b55cd2876ea3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.682390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.708175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.708781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.709056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.709319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.709976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:32.710271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.711678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.711786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8402d5-87ec-4945-ba83-66d374bcbb81 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.719379 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 38/79] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:32.720142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.747436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.748043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.748320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.748579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.771643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.818327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.819754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.824112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a7ff7-6424-40dd-99ae-f539517e67ad None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.857729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:32.858376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:32.858637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:32.858916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:32.860204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:32.862546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:32.868957 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 39/80] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:32.868957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.894723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:32.926612 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 42/81] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:32.927481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:32.973959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d3a52-eb72-4958-a4ea-e7c8ef6933e5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:32.989423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.007497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.008132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.008409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.008702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.018580 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:33.023930 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:33.024791 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:33.025609 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 40/82] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:33.026256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.030105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f5a85-c02e-47a4-adeb-b51737c68dd4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.063632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.064244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.064523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.064797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.065443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.066957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.088306 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 43/83] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:41:32 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:33.096295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.126993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.131575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970eff9a-42fd-4ef1-baf5-295f59826c19 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.175944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.176694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.176970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.177330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.178441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.180006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.186750 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 41/84] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:33.186750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.226331 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23e4f0-a576-4560-b98a-6a29984472bd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.234906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.268026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.268998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.268998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.269157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.300282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd859051-0afb-455e-be1d-045c1cdb216e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.341034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.341660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.341918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.342190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.343387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.345271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.346604 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:33.347706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.356801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.357753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.395579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.468874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278717f7-cfa6-4212-b1c9-d773213ae98d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.484556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.488811 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 44/86] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:33.490775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.514409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.515039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.515350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.515757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.516440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:33.516799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.518271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.528053 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:33.529000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.606661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f379c-2771-45cd-a3a9-b615a92175b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.641836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.652131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ace1d0-2486-4eec-909a-183264ec71e8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.652535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.653223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.653809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.654151 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.706662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.707362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.707643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.707915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.708650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=dd069fd5f27b41ce9e963cd415d9158d) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:33.708957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.710406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.718665 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 44/88] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/role_assignments?user.id=dd069fd5f27b41ce9e963cd415d9158d&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:33.728116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.754627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.756653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.827549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:33.915829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586a5c1-e4ac-4b6f-a38e-2c24ed4f69ef None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:33.956861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:33.957511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:33.957791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:33.958068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:33.958231 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 45/89] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:33.959136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:33.959256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:33.960823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:33.963622 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 45/90] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:33.964414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:34.007475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:34.086933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7b5c0-8887-4be6-9602-25b61055e43a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:34.124255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:34.125140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:34.125429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:34.126057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:34.127275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:34.129396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:34.131749 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 46/91] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:34.132793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:34.177292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5af7e-4aee-43c2-8d1a-5aafb7f221cb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:34.194087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:34.243662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:34.244507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:34.244784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:34.245055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:34.250933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:34.252497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:34.255342 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 46/92] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:33 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:34.256786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:34.308863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:34.322464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69dc2f-e6eb-4ab5-b6cb-b3f327f18daa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:34.357582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:34.358262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:34.358490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:34.358714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:34.359494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=cf7ec619bcef4004bcdaf4a26a07260b) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:34.359801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:34.361565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:34.371589 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 47/93] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:34 2022] GET /identity/v3/role_assignments?user.id=cf7ec619bcef4004bcdaf4a26a07260b&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:34.372491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:34.513123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c50655-eae2-4897-8c64-96a7a819fb8a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:34.555665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3de7d7-0d4e-4237-92a1-370ef32b8ba1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:34.595466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:34.596095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:34.596386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:34.596653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:34.616113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:34.616731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:34.617004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:34.617273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:34.618105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] RBAC: Adding query filter params (role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=dd069fd5f27b41ce9e963cd415d9158d) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:34.618409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:34.621519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:34.629692 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 48/94] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:34 2022] GET /identity/v3/role_assignments?user.id=dd069fd5f27b41ce9e963cd415d9158d&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=9056804b78cc417c8a175569edc8c78a => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:34.679205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:34.727210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:34.728719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:34.864251 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 47/95] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:34.869956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:34.914383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:35.016893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adf9e2-a39c-4da2-95f2-f95c9118431d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:35.056833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63644c-979c-4e36-96d9-ed82819952d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:35.057473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63644c-979c-4e36-96d9-ed82819952d1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:35.057744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63644c-979c-4e36-96d9-ed82819952d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:35.058010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63644c-979c-4e36-96d9-ed82819952d1 None admin] PATH_INFO: `/v3/projects/f393778ab07549cebefe53541131d1ee` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:35.061289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63644c-979c-4e36-96d9-ed82819952d1 None admin] RBAC: Authorizing `identity:get_project(project_id=f393778ab07549cebefe53541131d1ee)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:35.061289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63644c-979c-4e36-96d9-ed82819952d1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:35.063426 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 49/96] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:34 2022] GET /identity/v3/projects/f393778ab07549cebefe53541131d1ee => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:35.070621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:35.192081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:35.219388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba59912-b7ca-4f3c-b9af-d847db538424 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:35.254197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:35.254814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:35.255091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:35.255393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:35.256132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=f393778ab07549cebefe53541131d1ee, user.id=dd069fd5f27b41ce9e963cd415d9158d) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:35.256439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:35.257884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0603e7-bb5a-4a25-a207-5179d17d01df None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:35.263935 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:35 2022] GET /identity/v3/role_assignments?user.id=dd069fd5f27b41ce9e963cd415d9158d&scope.project.id=f393778ab07549cebefe53541131d1ee&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:35.881245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b291bab-172f-4736-874c-62efe6221750 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:35.882257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b291bab-172f-4736-874c-62efe6221750 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:35.882751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b291bab-172f-4736-874c-62efe6221750 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:35.884326 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 50/98] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:35.889939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e6e85-147f-444a-b84c-7f1b3b3d1b5c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:35.890353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e6e85-147f-444a-b84c-7f1b3b3d1b5c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:35.890520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e6e85-147f-444a-b84c-7f1b3b3d1b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:35.915994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0c9e6e85-147f-444a-b84c-7f1b3b3d1b5c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:35.917267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0c9e6e85-147f-444a-b84c-7f1b3b3d1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:36.000628 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 49/99] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:36.492680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d1ac9-ab82-41e9-bfbc-9140ee970e39 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:36.493353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d1ac9-ab82-41e9-bfbc-9140ee970e39 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:36.493827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d1ac9-ab82-41e9-bfbc-9140ee970e39 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:36.495447 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 51/100] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:36.501701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435de0f8-0ff3-47cc-aa71-ea59b9ae4ab9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:36.502353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435de0f8-0ff3-47cc-aa71-ea59b9ae4ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:36.502922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435de0f8-0ff3-47cc-aa71-ea59b9ae4ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:36.529511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-435de0f8-0ff3-47cc-aa71-ea59b9ae4ab9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:36.531073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-435de0f8-0ff3-47cc-aa71-ea59b9ae4ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:36.538099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d1ac9-ab82-41e9-bfbc-9140ee970e39 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:36.624291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:36.637495 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 50/101] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:36.638876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191905-1505-4bad-b009-1e40efcd2148 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:36.639157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191905-1505-4bad-b009-1e40efcd2148 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:36.639446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7191905-1505-4bad-b009-1e40efcd2148 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:36.640675 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 51/102] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:36.645672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:36.645941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:36.646208 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:36.677487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:36.678658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:36.721906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:36.738963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d1ac9-ab82-41e9-bfbc-9140ee970e39 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:36.794729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:36.795309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:36.795581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:36.795843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:36.797764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:36.799222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:36.800850 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 52/103] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:36.807059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:36.808579 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 52/104] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:36.929879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a44f9-abfb-44c0-921a-de7677ddbf52 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:36.975033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:36.976009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:36.976504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:36.976966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:36.977943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:36.979711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:37.029190 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 53/105] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:41:36 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:37.137645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:37.275728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cb0d6-e01e-4afe-9594-45db5c71f6ee None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:37.304811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:37.313792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:37.314464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.314738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.315007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.316538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:37.328085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:37.333968 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:37.334883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:37.335721 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 53/106] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:37.336377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:37.341321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:37.504932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1eba8d-2c89-4e1e-a5da-4308631e3b6b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:37.534244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbf90a-58ef-427c-9bfb-44b582531469 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:37.575997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:37.576654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.576931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.577200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.598208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:37.603115 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:37.604020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:37.604802 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 54/107] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:37.609997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:37.610483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:37.611113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.611415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.611684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.612358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:37.612657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:37.614102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:37.630584 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 54/108] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:37.636725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:37.702422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:37.720674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:37.760360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b764708-b86f-4c1f-bc5f-55efcc0dc384 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:37.779113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3822cf-2dba-4110-bf4f-4bb998bd84f4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:37.819700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:37.820342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.820618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.820775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e04ca-b12d-4ab2-af35-5796e67f2e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:37.820897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.821568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:37.821871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:37.822077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e04ca-b12d-4ab2-af35-5796e67f2e28 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.822521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e04ca-b12d-4ab2-af35-5796e67f2e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.823049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e04ca-b12d-4ab2-af35-5796e67f2e28 None admin] PATH_INFO: `/v3/users/cc0ffc7dd9ac49f09ac4269ada657b56` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.823382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e1ce-bb6c-4dd9-a915-68eb70d99954 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:37.833926 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 55/109] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:37.835323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385e1bb-10d8-46c4-90c8-fca935d95f44 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.835573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385e1bb-10d8-46c4-90c8-fca935d95f44 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.835848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385e1bb-10d8-46c4-90c8-fca935d95f44 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.842188 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 56/110] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:37.842931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385e1bb-10d8-46c4-90c8-fca935d95f44 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:37.854908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e04ca-b12d-4ab2-af35-5796e67f2e28 None admin] RBAC: Authorizing `identity:get_user(user_id=cc0ffc7dd9ac49f09ac4269ada657b56)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:37.856931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e04ca-b12d-4ab2-af35-5796e67f2e28 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:37.859270 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 55/111] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/users/cc0ffc7dd9ac49f09ac4269ada657b56 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:37.861427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:37.862179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:37.862653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:37.884127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:37.885663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:37.949668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385e1bb-10d8-46c4-90c8-fca935d95f44 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:37.966965 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 56/112] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:37.975199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.019300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245a873-e0ae-4268-bfb0-f09bb5864bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.020030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a873-e0ae-4268-bfb0-f09bb5864bfa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.020310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a873-e0ae-4268-bfb0-f09bb5864bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.020619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a873-e0ae-4268-bfb0-f09bb5864bfa None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.028022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a873-e0ae-4268-bfb0-f09bb5864bfa None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:38.031298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a873-e0ae-4268-bfb0-f09bb5864bfa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:38.033215 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 57/113] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:38.038374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47922b84-2629-4e0c-b527-e9a53ea94a0a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.038666 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47922b84-2629-4e0c-b527-e9a53ea94a0a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.038936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47922b84-2629-4e0c-b527-e9a53ea94a0a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.041267 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 58/114] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:38.049237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47922b84-2629-4e0c-b527-e9a53ea94a0a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.058422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:38.059946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:38.190146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6a12f-db17-4f08-9f60-4ace83e94a8b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:38.230460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.231140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.231497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.231770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.242923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47922b84-2629-4e0c-b527-e9a53ea94a0a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:38.278261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c74bb4-fd1f-4944-ad2c-417854fb4c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.278914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74bb4-fd1f-4944-ad2c-417854fb4c3c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.279210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74bb4-fd1f-4944-ad2c-417854fb4c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.279473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c74bb4-fd1f-4944-ad2c-417854fb4c3c None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.280575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74bb4-fd1f-4944-ad2c-417854fb4c3c None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:38.282305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c74bb4-fd1f-4944-ad2c-417854fb4c3c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:38.284192 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 59/115] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:38 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:38.285250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d467f4-5287-49bd-9fe1-9aa0efb4cde9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.285637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d467f4-5287-49bd-9fe1-9aa0efb4cde9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.285929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d467f4-5287-49bd-9fe1-9aa0efb4cde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.304079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b4d467f4-5287-49bd-9fe1-9aa0efb4cde9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:38.305182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b4d467f4-5287-49bd-9fe1-9aa0efb4cde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:38.306173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:38.307754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:38.351593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:38.378948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:38.391271 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 60/116] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:38.393742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f53c6-1dfb-4017-ab3e-6af1dbe5280a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.394241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f53c6-1dfb-4017-ab3e-6af1dbe5280a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.395449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f53c6-1dfb-4017-ab3e-6af1dbe5280a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.398223 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 61/117] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:38.399095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f53c6-1dfb-4017-ab3e-6af1dbe5280a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.439876 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 57/118] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:38.441167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.508757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f53c6-1dfb-4017-ab3e-6af1dbe5280a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:38.544549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a6bb27-3f97-4f4b-8371-8fb7f9e08c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.551212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6bb27-3f97-4f4b-8371-8fb7f9e08c03 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.551212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6bb27-3f97-4f4b-8371-8fb7f9e08c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.551212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a6bb27-3f97-4f4b-8371-8fb7f9e08c03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.552448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1edb-ece2-4aa4-b8f0-e70dfd49052e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:38.564890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a6bb27-3f97-4f4b-8371-8fb7f9e08c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d7a6bb27-3f97-4f4b-8371-8fb7f9e08c03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:38.570377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:38.571481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:38.572291 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:38.574835 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 62/119] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:38.576269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.576735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.577199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.593745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.594434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.594767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.595048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/dd069fd5f27b41ce9e963cd415d9158d/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.597151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] RBAC: Authorizing `identity:create_grant(project_id=01418d2a80e444dfa5e16569dd4fed3f, user_id=dd069fd5f27b41ce9e963cd415d9158d, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:38.598841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:38.599629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:38.601044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:38.609504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:38.615775 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 58/120] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:38 2022] PUT /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/dd069fd5f27b41ce9e963cd415d9158d/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:38.617894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.668436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:38.683130 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 63/121] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:38.688443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.779879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed0ede-83f8-4cf4-a8b0-d0c80291bb8e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:38.854759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.856069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.856069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.856069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.857475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:38.858957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:38.860820 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 59/122] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:38 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:38.861426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.882801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade22860-1aae-4c49-b9c1-eafc4e88b1e5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:38.926580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:38.927478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:38.927922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:38.928757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:38.929843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:38.930290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:38.932634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:38.947405 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 64/123] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:38.948471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:38.949369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.002458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee718e-8560-4f89-b95a-fade2b2ea757 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.020336 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.047503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.048680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.049098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.049667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.064385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:39.069318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486166-4e3d-43e0-b8c9-d32ed0c17e2d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:39.072920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:39.073936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:39.074911 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:39.077546 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 60/124] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:39.083662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.111223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.112105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.112989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.113488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.114644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] RBAC: Adding query filter params (role.id=253186fb50c242189536f642c4a53561, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=cc0ffc7dd9ac49f09ac4269ada657b56) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:39.115090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.117433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.124740 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 65/125] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:38 2022] GET /identity/v3/role_assignments?user.id=cc0ffc7dd9ac49f09ac4269ada657b56&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=253186fb50c242189536f642c4a53561 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:39.125399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.215616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08d897-7f57-41e6-a37a-6d30e56faeec None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.243953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.254360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.255334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.255773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.256179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.257530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.259316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.259511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d72a0f-2c58-469d-8ea0-5858a9b45de7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.261193 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 61/126] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:39.262069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.283710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.324003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.324679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.325103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.325382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.349271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:39.358258 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:39.359415 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:39.360387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:39.364259 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 66/127] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:39.371394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.434180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549123a-274f-46c9-8ddb-b3176f7de33c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.485315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f893c-3486-4a6a-aec8-9dd71c8ba3b1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.528919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.529555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.529833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.530108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.530790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:39.531098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.532518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.534526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.535252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.535530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.535803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.537785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.539285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.540941 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 67/128] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:39.541705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.551718 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 62/129] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:39.552444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.631014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.647300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.717515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07b7da3-65cc-44dc-a9e1-7df14dc1be0f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.721722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bd1c2-89ea-4322-9d3c-aeebcd0be8c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.753993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.754664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.754941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.755234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/dd069fd5f27b41ce9e963cd415d9158d/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.757654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] RBAC: Authorizing `identity:create_grant(project_id=01418d2a80e444dfa5e16569dd4fed3f, user_id=dd069fd5f27b41ce9e963cd415d9158d, role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.759168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.777959 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 63/130] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:39 2022] PUT /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/dd069fd5f27b41ce9e963cd415d9158d/roles/9056804b78cc417c8a175569edc8c78a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:39.779210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.791589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.792564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.793061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.793605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.794480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:39.803591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.805542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.819788 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 68/131] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:39.828529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:39.907936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac4d82-b369-4b91-9cf1-2402913f6385 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:39.928108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:39.951654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:39.952426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:39.952707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:39.952979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:39.958167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:39.958167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:39.958454 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 64/132] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:39.965338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:40.074753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4fba5-5fd2-4bec-822c-bc9df53d67c2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:40.085579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:40.123747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e705b-de03-4cc8-8664-dc8f1e7546ee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:40.140390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f78dff2-5b75-4bd4-98c2-7cd5f49ca4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:40.141050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78dff2-5b75-4bd4-98c2-7cd5f49ca4bb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.141330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78dff2-5b75-4bd4-98c2-7cd5f49ca4bb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.141608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78dff2-5b75-4bd4-98c2-7cd5f49ca4bb None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.143504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78dff2-5b75-4bd4-98c2-7cd5f49ca4bb None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:40.145099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f78dff2-5b75-4bd4-98c2-7cd5f49ca4bb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:40.151025 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 69/133] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:40.151025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031fb5-4ea9-489d-b470-c75c9ac76e01 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.151025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031fb5-4ea9-489d-b470-c75c9ac76e01 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.151025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c031fb5-4ea9-489d-b470-c75c9ac76e01 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.151623 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 70/134] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:40.153202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c031fb5-4ea9-489d-b470-c75c9ac76e01 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:40.181592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad95d4-3690-4f51-913c-fa59fbf9505f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:40.182637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad95d4-3690-4f51-913c-fa59fbf9505f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.183102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad95d4-3690-4f51-913c-fa59fbf9505f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.183626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad95d4-3690-4f51-913c-fa59fbf9505f None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.185724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad95d4-3690-4f51-913c-fa59fbf9505f None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:40.188107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad95d4-3690-4f51-913c-fa59fbf9505f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:40.190128 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 65/135] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:39 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:40.191994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.192517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.192977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.214521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:40.216140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:40.276926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c031fb5-4ea9-489d-b470-c75c9ac76e01 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:40.283535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:40.309445 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 66/136] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:40.315385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:40.336786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:40.337800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.338295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.338754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] PATH_INFO: `/v3/projects/f393778ab07549cebefe53541131d1ee` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.340094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] RBAC: Authorizing `identity:get_project(project_id=f393778ab07549cebefe53541131d1ee)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:40.341859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:40.343709 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 71/137] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:40 2022] GET /identity/v3/projects/f393778ab07549cebefe53541131d1ee => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:40.352256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:40.433199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e74b-8030-475a-a55c-d3fd7869a0b8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:40.478488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:40.479163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.479468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.479743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/cf7ec619bcef4004bcdaf4a26a07260b/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.483199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] RBAC: Authorizing `identity:create_grant(project_id=01418d2a80e444dfa5e16569dd4fed3f, user_id=cf7ec619bcef4004bcdaf4a26a07260b, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:40.483835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:40.490148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:40.511304 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 67/138] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:40 2022] PUT /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/cf7ec619bcef4004bcdaf4a26a07260b/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:40.596157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecd7b1-5673-4022-9b91-7dec4d5f4532 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:40.631887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:40.632579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.632852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.633121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] PATH_INFO: `/v3/projects/f393778ab07549cebefe53541131d1ee/users/dd069fd5f27b41ce9e963cd415d9158d/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.635317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] RBAC: Authorizing `identity:create_grant(project_id=f393778ab07549cebefe53541131d1ee, user_id=dd069fd5f27b41ce9e963cd415d9158d, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:40.636845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:40.664405 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 72/139] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:40 2022] PUT /identity/v3/projects/f393778ab07549cebefe53541131d1ee/users/dd069fd5f27b41ce9e963cd415d9158d/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:40.740283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:40.792285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:40.855986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bbc97-16dd-4cb0-874a-67f4211749da None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:40.893033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a735fc-bf5e-4c94-b284-04ae04275254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:40.893691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a735fc-bf5e-4c94-b284-04ae04275254 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:40.893994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a735fc-bf5e-4c94-b284-04ae04275254 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:40.894316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a735fc-bf5e-4c94-b284-04ae04275254 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:40.895525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a735fc-bf5e-4c94-b284-04ae04275254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:40.897003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a735fc-bf5e-4c94-b284-04ae04275254 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:40.898807 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 68/140] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:40.904493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:41.047005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b16245-04ef-451e-83d3-815e57892b4f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:41.048513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:41.096045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d3dc8-ad34-4245-93fc-b7de4a9f7430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:41.096670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3dc8-ad34-4245-93fc-b7de4a9f7430 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:41.096920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3dc8-ad34-4245-93fc-b7de4a9f7430 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:41.097186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d3dc8-ad34-4245-93fc-b7de4a9f7430 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:41.097919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3dc8-ad34-4245-93fc-b7de4a9f7430 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:41.099377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d3dc8-ad34-4245-93fc-b7de4a9f7430 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:41.134786 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 73/141] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:41:40 2022] POST /identity/v3/users => generated 283 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:41.797501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a5bb-a1bf-4cd4-ba16-cb83e90af84b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:41.797501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a5bb-a1bf-4cd4-ba16-cb83e90af84b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:41.797501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43a5bb-a1bf-4cd4-ba16-cb83e90af84b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:41.798668 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 69/142] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:41.804634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6132-a37d-457d-8eb1-7e280821e574 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:41.805208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6132-a37d-457d-8eb1-7e280821e574 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:41.805675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb6132-a37d-457d-8eb1-7e280821e574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:41.830505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-23eb6132-a37d-457d-8eb1-7e280821e574 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:41.832158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-23eb6132-a37d-457d-8eb1-7e280821e574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:41.905997 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 74/143] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:42.100752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523277d3-b413-409b-9265-203aab0defc0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.101410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523277d3-b413-409b-9265-203aab0defc0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.101971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523277d3-b413-409b-9265-203aab0defc0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:42.103582 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 70/144] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:42.109224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.110074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.110536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:42.133181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:42.134827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:42.222099 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 75/145] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:42.336309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523277d3-b413-409b-9265-203aab0defc0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:42.449748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523277d3-b413-409b-9265-203aab0defc0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:42.463529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:42.485587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:42.486278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.486561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.486837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:42.497571 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:42.502474 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:42.503356 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:42.504204 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 71/146] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:42.509747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:42.529475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:42.621217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ceb534-5490-41af-8a26-68a45068d22f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:42.656676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:42.657291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.657568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.657839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:42.658515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:42.658817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:42.660346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:42.669271 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 76/147] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:42.674853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:42.682869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:42.787891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05480d5-bbbe-4754-855a-a6cec55bcc6f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:42.827032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:42.827244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:42.827667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.827938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.828318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:42.829983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:42.831532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:42.833748 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 72/148] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:42 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:42.839424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:42.855200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a4d066-cc67-4357-86d0-79516e896a05 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:42.888388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:42.920230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:42.921318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.921812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.922268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:42.945636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:42.951702 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:42.952594 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:42.953377 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:42.956079 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 77/149] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:42.957638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51a4e0-f124-49da-a41c-30c6c0b91653 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:42.962579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:42.997212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:42.997840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:42.998153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:42.998421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.077031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.078660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.096569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9214d58-d268-46ce-a521-d0203f253ada None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:43.120374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:43.139659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:43.141132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.141599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.142053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.143016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:43.144384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.146117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.156467 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 78/150] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:43.163264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:43.167471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:43.190627 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 73/151] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:43.196594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:43.291611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f0249-93d0-49f3-95d0-5f4426de0643 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:43.336182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:43.337166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.337658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.338115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] PATH_INFO: `/v3/users/cc0ffc7dd9ac49f09ac4269ada657b56` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.341072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cc0ffc7dd9ac49f09ac4269ada657b56)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.342770 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.345553 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 79/152] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:43 2022] GET /identity/v3/users/cc0ffc7dd9ac49f09ac4269ada657b56 => generated 289 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:43.352115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:43.369325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a0666a-aad4-41e4-b630-9e4189b7cc74 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:43.434861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:43.435912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.436186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.436451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.437212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:43.437598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.439977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.440904 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 74/153] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:43 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:43.469500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:43.479123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2255fa1-d8f0-40f6-a13a-f87ff5e732d8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:43.483486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:43.519708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:43.521169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.521746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.522233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.523780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.526017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.527893 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 80/154] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:43 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:43.539522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:43.681543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb7619-213d-4653-96cc-d9873dc99776 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:43.726977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7dfa5-f7b0-4150-b481-f6f45ecdd1af None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:43.752990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:43.753763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.754111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.754387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.755146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=dd069fd5f27b41ce9e963cd415d9158d) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:43.755475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.756934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f20b2-fad0-4d84-8c49-ca90027a47b0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.771780 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 75/155] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:43 2022] GET /identity/v3/role_assignments?user.id=dd069fd5f27b41ce9e963cd415d9158d&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:43.798124 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0f9f-17fa-4143-8784-9e35aa0376f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:43.798806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0f9f-17fa-4143-8784-9e35aa0376f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.799074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0f9f-17fa-4143-8784-9e35aa0376f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.799371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0f9f-17fa-4143-8784-9e35aa0376f6 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/cc0ffc7dd9ac49f09ac4269ada657b56/roles/253186fb50c242189536f642c4a53561` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.804727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b0f9f-17fa-4143-8784-9e35aa0376f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=01418d2a80e444dfa5e16569dd4fed3f, user_id=cc0ffc7dd9ac49f09ac4269ada657b56, role_id=253186fb50c242189536f642c4a53561)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:43.806249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b0f9f-17fa-4143-8784-9e35aa0376f6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:43.834822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:43.841548 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 81/156] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:43 2022] PUT /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/users/cc0ffc7dd9ac49f09ac4269ada657b56/roles/253186fb50c242189536f642c4a53561 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:43.982815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366af53-3bf1-49ac-b397-55ba801d0bd0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.983438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366af53-3bf1-49ac-b397-55ba801d0bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.983438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366af53-3bf1-49ac-b397-55ba801d0bd0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:43.984981 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 76/157] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:43.990713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f7169-fbe0-439a-b723-43a45a5602a3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:43.991026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f7169-fbe0-439a-b723-43a45a5602a3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:43.991278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f7169-fbe0-439a-b723-43a45a5602a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:44.020421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e10f7169-fbe0-439a-b723-43a45a5602a3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:44.021638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e10f7169-fbe0-439a-b723-43a45a5602a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:44.101838 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 82/158] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:44.200853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2fd8f-c5b9-4561-a65f-0c5ed118e877 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:44.201196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2fd8f-c5b9-4561-a65f-0c5ed118e877 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:44.201435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2fd8f-c5b9-4561-a65f-0c5ed118e877 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:44.203083 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 77/159] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:44.209256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c73c4-9014-4acd-bf05-d76f90d15349 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:44.209780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c73c4-9014-4acd-bf05-d76f90d15349 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:44.209780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c73c4-9014-4acd-bf05-d76f90d15349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:44.236436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3c9c73c4-9014-4acd-bf05-d76f90d15349 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:44.237564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3c9c73c4-9014-4acd-bf05-d76f90d15349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:44.324571 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 83/160] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:44.667935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d35c-b2cf-47c0-a5d8-13e3222807fe None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:44.668338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d35c-b2cf-47c0-a5d8-13e3222807fe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:44.668550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d35c-b2cf-47c0-a5d8-13e3222807fe None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:44.670143 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 78/161] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:44.677249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:44.677526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:44.677796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:44.693115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d35c-b2cf-47c0-a5d8-13e3222807fe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:44.719132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:44.720335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:44.835535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:44.847323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:44.879954 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 84/162] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:44.896819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d35c-b2cf-47c0-a5d8-13e3222807fe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:44.910059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:44.980489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:44.981130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:44.981434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:44.981697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.008252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:45.013208 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:45.014132 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:45.014971 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 79/163] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:41:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:45.022901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.108366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1d7f8-0778-424b-8eae-cf7dfabc5409 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:45.166128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:45.166785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:45.167054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:45.167747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.187108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:45.193126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:45.194064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:45.195447 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 85/164] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:45.200159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.262145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee0eee-9c15-4d90-a452-573fcb49fd21 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:45.314910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:45.324882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:45.325508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:45.325780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:45.326048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.326722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:45.327015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:45.332769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:45.339525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:45.345754 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 80/165] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:45.351842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.425404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79a84e0-e9d1-4b72-99f1-7e5569c18d4d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:45.476602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:45.477600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:45.478083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:45.478566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.479483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:45.479992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:45.481592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:45.495326 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 86/166] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:45.496510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.534869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318ac28-02fa-4290-82e0-198f50d488d3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:45.640941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:45.641587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:45.641866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:45.642121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.647890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:45.649362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:45.651210 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 81/167] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:45.651765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.681453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68b207-37c6-4d13-87de-3784511c0b86 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:45.704358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:45.730667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:45.731354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:45.731637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:45.731911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.742871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:45.749184 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:45.750196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 87/168] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:45.751400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.785403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:45.850418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a5ae4-c13a-4276-8635-3e088fba5ad7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:45.904863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:45.905512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:45.905779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:45.906045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] PATH_INFO: `/v3/users/cf7ec619bcef4004bcdaf4a26a07260b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:45.911864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] RBAC: Authorizing `identity:get_user(user_id=cf7ec619bcef4004bcdaf4a26a07260b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:45.913340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:45.914943 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 82/169] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/users/cf7ec619bcef4004bcdaf4a26a07260b => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:45.915734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:45.986283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d07d6-3476-470d-b5b9-9c72c11e7bc3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.046951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f8382-12f9-4861-9ef4-1e9a45f6bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.048059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f8382-12f9-4861-9ef4-1e9a45f6bf2e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.048746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f8382-12f9-4861-9ef4-1e9a45f6bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.049237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f8382-12f9-4861-9ef4-1e9a45f6bf2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.051546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e930371-39bd-4f0e-a3d0-b9841bba7c72 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.079079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:46.087740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.088645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.089329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.089789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.090654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:46.091152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:46.092744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:46.101606 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 83/170] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:46.102721 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:46.130088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:46.192190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f8382-12f9-4861-9ef4-1e9a45f6bf2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:46.193684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f8382-12f9-4861-9ef4-1e9a45f6bf2e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:46.218192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a018a-8b86-42c4-8f52-6ce2b0bcfc7d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.266053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.266667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.266940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.267231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.360493 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 88/171] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:46.365826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613deb-7ddf-4ea8-b352-3a09a85ee685 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.366353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613deb-7ddf-4ea8-b352-3a09a85ee685 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.366771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b613deb-7ddf-4ea8-b352-3a09a85ee685 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.368337 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 89/172] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:46.369551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613deb-7ddf-4ea8-b352-3a09a85ee685 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:46.382958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:46.403399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:46.404892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:46.502724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:46.547663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b613deb-7ddf-4ea8-b352-3a09a85ee685 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.592713 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 84/173] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:46.592713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:46.596957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f172cba-e8bf-47b8-b724-953192010688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.597792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f172cba-e8bf-47b8-b724-953192010688 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.598220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f172cba-e8bf-47b8-b724-953192010688 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.598643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f172cba-e8bf-47b8-b724-953192010688 None admin] PATH_INFO: `/v3/users/dd069fd5f27b41ce9e963cd415d9158d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.600460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f172cba-e8bf-47b8-b724-953192010688 None admin] RBAC: Authorizing `identity:get_user(user_id=dd069fd5f27b41ce9e963cd415d9158d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:46.602436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f172cba-e8bf-47b8-b724-953192010688 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:46.604828 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 90/174] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:46 2022] GET /identity/v3/users/dd069fd5f27b41ce9e963cd415d9158d => generated 275 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:46.606356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c53b-6cc0-4ad8-94e8-f81536abece5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.606778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c53b-6cc0-4ad8-94e8-f81536abece5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.607594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c53b-6cc0-4ad8-94e8-f81536abece5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.625923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eb90c53b-6cc0-4ad8-94e8-f81536abece5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:46.627121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eb90c53b-6cc0-4ad8-94e8-f81536abece5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:46.688138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:46.697295 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 91/175] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:46.700554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881bd92-87be-46ff-9a0f-72332c8f07bd None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.700833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881bd92-87be-46ff-9a0f-72332c8f07bd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.701098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881bd92-87be-46ff-9a0f-72332c8f07bd None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.706482 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 92/176] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:46.706482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881bd92-87be-46ff-9a0f-72332c8f07bd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:46.708987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa531a4c-7bc1-468d-9559-8404f322eb13 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.748103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69bc1c2-a065-4dfb-9604-9a2253d91cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.749549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bc1c2-a065-4dfb-9604-9a2253d91cef None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.750054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bc1c2-a065-4dfb-9604-9a2253d91cef None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.750532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69bc1c2-a065-4dfb-9604-9a2253d91cef None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.752701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bc1c2-a065-4dfb-9604-9a2253d91cef None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:46.755105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69bc1c2-a065-4dfb-9604-9a2253d91cef None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:46.757809 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 85/177] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:46 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:46.759603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070525be-79ca-40f0-9e69-52afcafeca23 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.760086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070525be-79ca-40f0-9e69-52afcafeca23 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.760543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070525be-79ca-40f0-9e69-52afcafeca23 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.762519 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 86/178] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:46.763683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070525be-79ca-40f0-9e69-52afcafeca23 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:46.776839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:46.815544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881bd92-87be-46ff-9a0f-72332c8f07bd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.872857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4dc95-72ed-45a7-b07a-28a3bd451e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.873680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dc95-72ed-45a7-b07a-28a3bd451e3d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.873946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dc95-72ed-45a7-b07a-28a3bd451e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.874321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4dc95-72ed-45a7-b07a-28a3bd451e3d None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.875548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4dc95-72ed-45a7-b07a-28a3bd451e3d None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:46.877118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4dc95-72ed-45a7-b07a-28a3bd451e3d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:46.878949 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 93/179] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:46 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:46.880354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.880697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.881027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:46.907015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:46.908308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:46.920290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070525be-79ca-40f0-9e69-52afcafeca23 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:46.968402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:46.969004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:46.969357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:46.969692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.007211 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 94/180] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:47.007211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.009866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.054441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.056021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:47.074252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.111401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9bcc3d-b291-4c0a-82a7-41664fc30df0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.145614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.146260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.150311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.150311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.150311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] RBAC: Adding query filter params (role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=dd069fd5f27b41ce9e963cd415d9158d) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:47.150311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.150311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f9ae-f578-49c4-9715-927ab2b35649 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:47.156390 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 95/181] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/role_assignments?user.id=dd069fd5f27b41ce9e963cd415d9158d&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=9056804b78cc417c8a175569edc8c78a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:47.164490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.164772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.165042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.172954 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 87/182] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:47.172954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.190582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:47.191905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:47.277072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.278641 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 96/183] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:47.279582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.332921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3584013-a993-43d5-a60a-93e80b1144c9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.358553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.374808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.375953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.375953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.376117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.376862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=cf7ec619bcef4004bcdaf4a26a07260b) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:47.377166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.378660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4decb-f0b4-4705-ac80-fdf2f1533590 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:47.385774 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 88/184] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/role_assignments?user.id=cf7ec619bcef4004bcdaf4a26a07260b&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:47.391248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df7327-daad-4003-b962-e00de0518869 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.391248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df7327-daad-4003-b962-e00de0518869 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.391248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df7327-daad-4003-b962-e00de0518869 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.391248 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 89/185] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:47.391248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df7327-daad-4003-b962-e00de0518869 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.422496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2e812-96d0-4515-a8ed-f6bbb15cbd43 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.477295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.477671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.478098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.478537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.494374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:47.500480 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:47.501481 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:47.502273 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 97/186] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:47.502944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.547677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df7327-daad-4003-b962-e00de0518869 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.581002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.600779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06ad2d-199e-4768-ae56-a3fe3d140d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.601450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06ad2d-199e-4768-ae56-a3fe3d140d43 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.601708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06ad2d-199e-4768-ae56-a3fe3d140d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.601973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06ad2d-199e-4768-ae56-a3fe3d140d43 None admin] PATH_INFO: `/v3/projects/f393778ab07549cebefe53541131d1ee` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.603924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06ad2d-199e-4768-ae56-a3fe3d140d43 None admin] RBAC: Authorizing `identity:get_project(project_id=f393778ab07549cebefe53541131d1ee)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.606303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06ad2d-199e-4768-ae56-a3fe3d140d43 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:47.608247 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 90/187] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/projects/f393778ab07549cebefe53541131d1ee => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:47.609202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.609465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.609756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.628416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:47.630078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:47.630911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650c635-10d3-405b-9b86-c12915cc099d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.654242 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.670521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.671160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.671463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.671798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.683307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:47.688444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:47.689427 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 98/188] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:47.690422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.713806 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 91/189] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:47.714641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.793254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edc165-7fa3-432c-86d9-715854e13a77 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.821848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d7f5ba-7a62-4f8e-920b-b6091ac015b6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.826699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.827337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.827609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.827869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.828528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:47.828821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.830598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:47.839204 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 99/190] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:47.840029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.846558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.863501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.864101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.864416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.864685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.865808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.867265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:47.868829 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 92/191] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:47.869565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:47.894351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:47.957648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc8c6d-7226-4645-91f3-69c7de061268 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.971733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ef5b6-a29d-4c26-8686-1388b948c79d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:47.991135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:47.991760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:47.992035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:47.992299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:47.993015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=f393778ab07549cebefe53541131d1ee, user.id=dd069fd5f27b41ce9e963cd415d9158d) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:47.993314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:47.994760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.000365 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 100/192] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/role_assignments?user.id=dd069fd5f27b41ce9e963cd415d9158d&scope.project.id=f393778ab07549cebefe53541131d1ee&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:48.001193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.008038 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.008682 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.008953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.009218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.010336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:48.010336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.011681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.026002 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 93/193] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:48.026002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.081514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.122933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d14f09-2b68-42de-9e1a-8d21ae7af974 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.130507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.146599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcec294-33f5-42de-866f-2aaba92218b3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.164460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.165205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.165542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.165875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] PATH_INFO: `/v3/users/8355c81404214834ab08c5919abf4073` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.181239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] RBAC: Authorizing `identity:get_user(user_id=8355c81404214834ab08c5919abf4073)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.182842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.184675 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 101/194] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/users/8355c81404214834ab08c5919abf4073 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:48.186021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.189041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.189903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.190182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.190452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.191249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.193077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.224945 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 94/195] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:41:48 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:48.225522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.319830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98609-8d86-436d-b531-4af15e495e09 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.356074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c59ab6-e227-4901-b88d-42790e468e32 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.362584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.363566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.363982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.364432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.364877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.376793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:48.381836 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:48.382827 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:48.383703 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 102/196] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:48.385717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.406584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.407263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.407540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.407814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] PATH_INFO: `/v3/users/8355c81404214834ab08c5919abf4073` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.408973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] RBAC: Authorizing `identity:get_user(user_id=8355c81404214834ab08c5919abf4073)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.410517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.411919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.412783 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 95/197] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/users/8355c81404214834ab08c5919abf4073 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:48.413676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.505595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2068321-f092-4436-a783-fe93d3a4d571 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.529845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339222d-bbac-4e86-aad4-49ead1c6b590 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.550374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.552090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.552454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.552734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.577228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.577855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.580073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.580073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.580073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:48.580073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.581206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.592701 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 96/198] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:48.594149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.637452 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.638940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.647307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.664889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.733657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40350c0a-fc26-4af7-a0dc-1591b39b20a2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.744235 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 103/199] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:48.745089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.774594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.775696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.776500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.777155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.849494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6f1ab-cae2-466b-b5c5-fdce88727c8b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:48.863225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.865076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.874491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.886546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:48.887204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:48.887472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:48.887749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] PATH_INFO: `/v3/users/cc0ffc7dd9ac49f09ac4269ada657b56` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:48.888999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] RBAC: Authorizing `identity:get_user(user_id=cc0ffc7dd9ac49f09ac4269ada657b56)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:48.890353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:48.892070 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 104/200] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/users/cc0ffc7dd9ac49f09ac4269ada657b56 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:48.892833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:48.909011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:48.992051 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 97/201] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:48.993256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:49.002506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2f49a-8373-41de-9a64-e05436bd0b14 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.037673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.038223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.038494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.038773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.039909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.041404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.043542 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 105/202] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:49.044405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:49.098158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea01f94-9faf-4d77-ba9b-3a4a3f27417a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.106429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:49.135097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.136132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.136595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.137051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.140970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:49.149262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d2961-b73a-492b-859b-07b133e5c3c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.186001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.186691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.186970 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.187300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.188481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.190328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.192240 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 106/203] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:49 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:49.193104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:49.213153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.214988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.303040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e80460-8098-42ad-818f-8a42f4b2bb78 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.320664 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 98/204] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:49.321926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:49.335577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:49.345645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.346317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.346594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.346858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.347673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=8355c81404214834ab08c5919abf4073) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:49.347994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.349559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.355776 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 107/205] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:49 2022] GET /identity/v3/role_assignments?user.id=8355c81404214834ab08c5919abf4073&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:49.356648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:49.361345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:49.432768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b65577-61db-4bdb-93f2-397bccd69a68 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.475207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66929325-f84f-4911-b872-e9b5a022aa48 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.475456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.475710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.475988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.476256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.477099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] RBAC: Adding query filter params (role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=8355c81404214834ab08c5919abf4073) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:49.477409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.478889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.493543 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 99/206] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:49 2022] GET /identity/v3/role_assignments?user.id=8355c81404214834ab08c5919abf4073&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=9056804b78cc417c8a175569edc8c78a => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:49.518705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e8a4c8-532b-4fb1-ab8f-817d7c2a54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.519446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8a4c8-532b-4fb1-ab8f-817d7c2a54b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.519722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8a4c8-532b-4fb1-ab8f-817d7c2a54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.519991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e8a4c8-532b-4fb1-ab8f-817d7c2a54b7 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.521288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8a4c8-532b-4fb1-ab8f-817d7c2a54b7 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.522811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e8a4c8-532b-4fb1-ab8f-817d7c2a54b7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.524626 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 108/207] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:49 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:49.530861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:49.660094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:49.663885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f98ba-6519-421d-9096-da081a3d7867 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:49.709379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:49.710425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:49.710890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:49.711380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:49.712317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] RBAC: Adding query filter params (role.id=253186fb50c242189536f642c4a53561, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f, user.id=cc0ffc7dd9ac49f09ac4269ada657b56) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:49.712829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:49.714499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7424b-92af-413a-9b97-4f988ca7da9c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:49.722086 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 100/208] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:49 2022] GET /identity/v3/role_assignments?user.id=cc0ffc7dd9ac49f09ac4269ada657b56&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=253186fb50c242189536f642c4a53561 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:50.778066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170d25c-7270-43ff-a5c6-89d6ca827e7b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:50.778066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170d25c-7270-43ff-a5c6-89d6ca827e7b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:50.778066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170d25c-7270-43ff-a5c6-89d6ca827e7b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:50.779447 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 109/209] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:50.790538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427a9e8-6149-4281-bcee-9c4dafa1706b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:50.790538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427a9e8-6149-4281-bcee-9c4dafa1706b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:50.790538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9427a9e8-6149-4281-bcee-9c4dafa1706b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:50.807882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9427a9e8-6149-4281-bcee-9c4dafa1706b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:50.809383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9427a9e8-6149-4281-bcee-9c4dafa1706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:50.877622 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 101/210] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:50.885965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c441ef-a8e8-409f-a004-5e7d5c0d8419 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:50.886479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c441ef-a8e8-409f-a004-5e7d5c0d8419 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:50.886957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c441ef-a8e8-409f-a004-5e7d5c0d8419 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:50.888839 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 110/211] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:50.896666 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:50.897181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:50.897638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:50.916381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:50.917850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:50.987112 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 102/212] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:51.145905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c441ef-a8e8-409f-a004-5e7d5c0d8419 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:51.207777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:51.254981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c441ef-a8e8-409f-a004-5e7d5c0d8419 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:51.261455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:51.289475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:51.293398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:51.294155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.294543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.294818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.296786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:51.298221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:51.300346 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 111/213] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:51.305683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:51.369036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0da1-d97a-474d-b73d-ecdc064ea22f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:51.408081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:51.409288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.409638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.410049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.414949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4cb3a-06de-44a4-9d2e-4fa6d9061c6a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:51.421668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:51.428047 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:51.428953 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:51.429762 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 103/214] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:51.435487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:51.447039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:51.457215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:51.458183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.458778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.459262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.460141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:51.461895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:51.495444 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 112/215] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:41:51 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:51.530638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:51.548158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d773dd-0d6d-4562-9fb3-6137881c994e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:51.586629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:51.587278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.587599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.587889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.588553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:51.588859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:51.590256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:51.600229 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 104/216] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:51.606222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:51.723857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25189310-ef03-4baa-b5eb-ab7c8537b23c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:51.758872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63248524-f161-4491-8d2c-1fa11782a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:51.759931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248524-f161-4491-8d2c-1fa11782a4fc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.760385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248524-f161-4491-8d2c-1fa11782a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.760830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63248524-f161-4491-8d2c-1fa11782a4fc None admin] PATH_INFO: `/v3/users/d64fc3014247446aa6c088af3ffdc7d8` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.777233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63248524-f161-4491-8d2c-1fa11782a4fc None admin] RBAC: Authorizing `identity:get_user(user_id=d64fc3014247446aa6c088af3ffdc7d8)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:51.779271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63248524-f161-4491-8d2c-1fa11782a4fc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:51.781750 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 113/217] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:51 2022] GET /identity/v3/users/d64fc3014247446aa6c088af3ffdc7d8 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:51.788045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:51.904791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0b67ba-a5da-49d9-93d4-a3b92eef4749 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:51.905995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1ee94-ad7a-47cc-a1fb-b5415321564f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.906694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1ee94-ad7a-47cc-a1fb-b5415321564f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.907161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1ee94-ad7a-47cc-a1fb-b5415321564f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.910128 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 114/218] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:51.915447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0d696-68f2-41ea-9e28-a0e3fcc96576 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.915717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0d696-68f2-41ea-9e28-a0e3fcc96576 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.915984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab0d696-68f2-41ea-9e28-a0e3fcc96576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.941820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3ab0d696-68f2-41ea-9e28-a0e3fcc96576 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:51.943080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3ab0d696-68f2-41ea-9e28-a0e3fcc96576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:51.949803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:51.950731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:51.951245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:51.951711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:51.970939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:51.988235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:52.024656 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 115/219] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:52.026259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d5269-7b90-4a3d-9373-1f3f14e98c12 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.026735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d5269-7b90-4a3d-9373-1f3f14e98c12 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.027231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d5269-7b90-4a3d-9373-1f3f14e98c12 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.029109 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 116/220] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:52.035232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:52.035413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.035495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:52.035565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.035565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.054220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:52.055459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:52.128557 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 117/221] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:52.143990 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 105/222] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:52.149954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:52.261563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8fc62-cd1f-4263-bab9-03b6749126f3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:52.263068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:52.298834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:52.299507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.299779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.300043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.301272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:52.302758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:52.304725 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 118/223] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:52.310646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:52.374213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:52.412001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:52.445824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346803d8-a42d-4352-b5e0-acd2c8117180 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:52.505371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:52.505982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.506255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.506524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.507284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=d64fc3014247446aa6c088af3ffdc7d8) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:52.507593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:52.509030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:52.515868 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 106/224] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/role_assignments?user.id=d64fc3014247446aa6c088af3ffdc7d8&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:52.516641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:52.552624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9434853-e28b-45ae-ac1c-79fd793fccb4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:52.602236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:52.602893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.603165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.607136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.631824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:52.636742 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:52.637656 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:52.638855 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 119/225] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:52.648961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:52.719262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b0bde-fe74-4456-bf06-cac718957834 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:52.732416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:52.745823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:52.764932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:52.766449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.766936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.767463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.781570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:52.790452 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:52.791399 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:52.792216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:52.793502 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 107/226] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:52.798938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:52.845720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a501de-d6b8-4ee6-87c2-4188a2d02ea3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:52.884466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:52.885553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:52.886036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:52.886535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:52.887680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:52.888177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:52.889993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:52.900716 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 120/227] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:52.905756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:53.009369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0db2-a8ac-4e72-9917-4d9c767a10f6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.010241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:53.022103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb78dcf-6f08-4447-b4ab-3d022220962e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.061168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.062264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.062727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.063270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] PATH_INFO: `/v3/users/8355c81404214834ab08c5919abf4073` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.065346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] RBAC: Authorizing `identity:get_user(user_id=8355c81404214834ab08c5919abf4073)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.067623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.070456 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 121/228] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/users/8355c81404214834ab08c5919abf4073 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:53.075907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:53.095903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.096535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.096817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.097088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.097807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:53.098099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.099529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.108306 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 108/229] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:41:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:53.114003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:53.151084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:53.217865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d68db-b111-428b-8db3-504d9e196839 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.238777 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121252c-f43a-4d02-a773-474887c538f6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.257490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.258434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.258896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.259389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.261181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.262829 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.265753 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 122/230] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:53 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:53.272822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:53.288044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.288683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.288956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.289224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] PATH_INFO: `/v3/users/8355c81404214834ab08c5919abf4073` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.290424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] RBAC: Authorizing `identity:get_user(user_id=8355c81404214834ab08c5919abf4073)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.291893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.297962 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 109/231] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:53 2022] GET /identity/v3/users/8355c81404214834ab08c5919abf4073 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:53.299396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:53.343467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:53.398849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f9856-f6ac-43be-ad13-cf4b6100ac9b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.401539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:53.433344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.433975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.434249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.434517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.436015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.437445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.439414 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 110/232] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:53 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:53.445037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:53.506483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176f17f-ac36-41c9-a20d-34a1d704e032 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.543679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9925785-d88f-4074-986d-fd9183e998c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.544301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9925785-d88f-4074-986d-fd9183e998c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.544576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9925785-d88f-4074-986d-fd9183e998c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.544890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9925785-d88f-4074-986d-fd9183e998c8 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/8355c81404214834ab08c5919abf4073/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.551147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9925785-d88f-4074-986d-fd9183e998c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d86b1548e5624d668a2cd70a520c24f5, user_id=8355c81404214834ab08c5919abf4073, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.552682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9925785-d88f-4074-986d-fd9183e998c8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.585052 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 123/233] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:53 2022] PUT /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/8355c81404214834ab08c5919abf4073/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:53.605249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91048608-9fe1-4b40-aa9f-2f605c83d9a9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:53.669742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c72102-d860-4b07-b51c-33f693ac68e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:53.670424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72102-d860-4b07-b51c-33f693ac68e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:53.670700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72102-d860-4b07-b51c-33f693ac68e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:53.670967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c72102-d860-4b07-b51c-33f693ac68e8 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/8355c81404214834ab08c5919abf4073/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:53.673219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72102-d860-4b07-b51c-33f693ac68e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d86b1548e5624d668a2cd70a520c24f5, user_id=8355c81404214834ab08c5919abf4073, role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:53.674675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c72102-d860-4b07-b51c-33f693ac68e8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:53.694900 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 111/234] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:53 2022] PUT /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/8355c81404214834ab08c5919abf4073/roles/9056804b78cc417c8a175569edc8c78a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:41:54.192953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b4198-eea2-45cf-ae2c-6c95176bc6d1 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:54.193932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b4198-eea2-45cf-ae2c-6c95176bc6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:54.194396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b4198-eea2-45cf-ae2c-6c95176bc6d1 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:54.195968 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 124/235] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:54.201536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959ae21-bd98-40f6-bb37-08fa88dd168c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:54.202337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959ae21-bd98-40f6-bb37-08fa88dd168c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:54.202810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959ae21-bd98-40f6-bb37-08fa88dd168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:54.221124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d959ae21-bd98-40f6-bb37-08fa88dd168c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:54.222791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d959ae21-bd98-40f6-bb37-08fa88dd168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:54.245482 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:54.299329 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 112/236] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:54.480782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9c26f-dfaf-46e7-b820-3135c32f7b7c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:54.480950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9c26f-dfaf-46e7-b820-3135c32f7b7c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:54.481211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9c26f-dfaf-46e7-b820-3135c32f7b7c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:54.482701 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 125/237] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:54.488104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:54.488381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:54.488647 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:54.512856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:54.514040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:54.582475 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 113/238] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:54.620628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9c26f-dfaf-46e7-b820-3135c32f7b7c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:54.711692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:54.726397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9c26f-dfaf-46e7-b820-3135c32f7b7c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:54.761725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:54.762277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:54.762545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:54.762860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:54.774287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:54.779126 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:54.780072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:54.780881 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:54.783560 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 126/239] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:54.786732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:54.886684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:54.904475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd31f9-07e2-44f3-9b9f-eccc65883387 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:54.953368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:54.954041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:54.954313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:54.954581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:54.955349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:54.955645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:54.957055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:54.968458 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 114/240] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:54.974259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:55.057790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:55.116390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd335d42-aba4-45f9-999a-3075539fa323 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:55.151652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:55.152793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.153265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.153728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] PATH_INFO: `/v3/users/056c163c883245639af749a75aa0e9ea` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.168595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] RBAC: Authorizing `identity:get_user(user_id=056c163c883245639af749a75aa0e9ea)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:55.170291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:55.172103 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 127/241] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:54 2022] GET /identity/v3/users/056c163c883245639af749a75aa0e9ea => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:55.180864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:55.195544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:55.255145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dca189-c6f2-4b76-9039-8395b41a148c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:55.288589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2447375-2819-45aa-8a44-40cb7c4874df None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:55.288806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c1333-adb8-43ce-bdc5-a9d297ba6673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:55.289425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c1333-adb8-43ce-bdc5-a9d297ba6673 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.289697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c1333-adb8-43ce-bdc5-a9d297ba6673 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.289989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c1333-adb8-43ce-bdc5-a9d297ba6673 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.291267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c1333-adb8-43ce-bdc5-a9d297ba6673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:55.292707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c1333-adb8-43ce-bdc5-a9d297ba6673 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:55.294264 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 115/242] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:41:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:55.299888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6004aa-c1c2-4a76-a862-4e0ba988f0ac None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.300165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6004aa-c1c2-4a76-a862-4e0ba988f0ac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.300487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6004aa-c1c2-4a76-a862-4e0ba988f0ac None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.301765 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 116/243] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:55.303257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6004aa-c1c2-4a76-a862-4e0ba988f0ac None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:55.323469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:55.326217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:55.326838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.327283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.327566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.410645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6004aa-c1c2-4a76-a862-4e0ba988f0ac None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:55.417385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:55.419147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:55.435613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:55.450269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1966c30-5093-45eb-9065-8979df7f8124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:55.451110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966c30-5093-45eb-9065-8979df7f8124 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.451238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966c30-5093-45eb-9065-8979df7f8124 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.451621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1966c30-5093-45eb-9065-8979df7f8124 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.452282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1966c30-5093-45eb-9065-8979df7f8124 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:55.453731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1966c30-5093-45eb-9065-8979df7f8124 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:55.482996 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 117/244] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:41:55 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:41:55.484487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.484773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.485080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.508060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:55.509342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:55.535841 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 128/245] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:41:55.541123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:55.592099 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 118/246] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:55.663831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77a3f5-129c-4988-9ee9-5f1876e0d6f2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:55.672739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:55.704842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:55.705798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.706246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.706827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.708249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:55.709917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:55.712104 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 129/247] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:55 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:55.719242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:55.721528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:55.822069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3994c1-7003-498d-93fb-3ed475d282a2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:55.856509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:55.857034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:55.857304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:55.857569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:55.858297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] RBAC: Adding query filter params (role.id=253186fb50c242189536f642c4a53561, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=056c163c883245639af749a75aa0e9ea) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:55.858595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:55.860068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:55.865963 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 119/248] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:55 2022] GET /identity/v3/role_assignments?user.id=056c163c883245639af749a75aa0e9ea&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=253186fb50c242189536f642c4a53561 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:55.923052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:56.041159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33deb4-000f-4ac6-84c4-7418bd90a40b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:56.085645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:56.086331 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.086607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.086936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.098866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:56.103693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:56.104593 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:56.105411 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 130/249] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:56.111145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:56.193509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:56.231816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33b6c64-4b43-43be-9954-ce2966475c7f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:56.270587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:56.271372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.271554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.271865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.272543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:56.272830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:56.274272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:56.287269 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 120/250] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:56.289256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:56.328966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:56.407658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b265549-1477-4369-91b8-cc2a3d26aa5f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:56.445022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f27b5-d7a6-4b87-9962-36b1d1b83bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:56.445658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f27b5-d7a6-4b87-9962-36b1d1b83bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.445927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f27b5-d7a6-4b87-9962-36b1d1b83bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.446190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f27b5-d7a6-4b87-9962-36b1d1b83bd3 None admin] PATH_INFO: `/v3/users/d64fc3014247446aa6c088af3ffdc7d8` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.447352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f27b5-d7a6-4b87-9962-36b1d1b83bd3 None admin] RBAC: Authorizing `identity:get_user(user_id=d64fc3014247446aa6c088af3ffdc7d8)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:56.448894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f27b5-d7a6-4b87-9962-36b1d1b83bd3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:56.450659 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 131/251] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:56 2022] GET /identity/v3/users/d64fc3014247446aa6c088af3ffdc7d8 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:56.456441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:56.501320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aed2c-14df-4143-8c0d-08004c924c11 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.501667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aed2c-14df-4143-8c0d-08004c924c11 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.501848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aed2c-14df-4143-8c0d-08004c924c11 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.503342 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 132/252] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:56.508982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.509261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.509528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.542149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:56.544709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:56.608720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bca4ce-7b83-4da8-9883-4ead72523bdb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:56.667978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:56.673473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce29e3-e7ca-42b2-af8b-230fb40183bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:56.674115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce29e3-e7ca-42b2-af8b-230fb40183bc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.674389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce29e3-e7ca-42b2-af8b-230fb40183bc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.674657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce29e3-e7ca-42b2-af8b-230fb40183bc None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.676714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce29e3-e7ca-42b2-af8b-230fb40183bc None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:56.677270 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 133/253] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:56.678600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce29e3-e7ca-42b2-af8b-230fb40183bc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:56.680516 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 121/254] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:56 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:56.687504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:56.718387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22ff73-cbbe-4016-935d-7f9a1b09fd1d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.718701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22ff73-cbbe-4016-935d-7f9a1b09fd1d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.718926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22ff73-cbbe-4016-935d-7f9a1b09fd1d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.720423 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 122/255] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:41:56.727871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:56.732552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.732825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.733094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.763626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:56.764829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:56.901322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809f0b2-69bc-4bb2-ae47-c06063eb67f6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:56.913932 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 123/256] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:56.956234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:56.956874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:56.957147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:56.957414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/d64fc3014247446aa6c088af3ffdc7d8/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:56.959351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] RBAC: Authorizing `identity:create_grant(project_id=d86b1548e5624d668a2cd70a520c24f5, user_id=d64fc3014247446aa6c088af3ffdc7d8, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:56.960800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:56.977049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:56.986156 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 134/257] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:41:56 2022] PUT /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/d64fc3014247446aa6c088af3ffdc7d8/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:41:57.125310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13efd5-06b8-4095-80fc-0a8f56a8cc48 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:57.165778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:57.177870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:57.178522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:57.178798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:57.179066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:57.196164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:57.201289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:57.202168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:57.203378 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 124/258] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:41:57.210465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:57.298349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:57.347716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:57.357916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59030-b9d3-46c7-847e-b64535da2f58 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:57.399501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:57.400382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:57.400789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:57.401517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:57.402317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:57.403059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:57.404607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:57.415706 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 135/259] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:57.421055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:57.432927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6769046-cd73-47f0-b6e7-d223554f5c0a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:57.485882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:57.486850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:57.488090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:57.488498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:57.522296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:57.530504 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:57.532923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:41:57.533760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:41:57.538649 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 125/260] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:41:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:57.552028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:57.637409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf968d38-3c8e-4fe4-ab45-7dff32403986 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:57.668508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:57.712978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:57.713862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:57.714273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:57.714674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] PATH_INFO: `/v3/users/8355c81404214834ab08c5919abf4073` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:57.715983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=8355c81404214834ab08c5919abf4073)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:57.717588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:57.720376 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 136/261] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:57 2022] GET /identity/v3/users/8355c81404214834ab08c5919abf4073 => generated 283 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:57.725074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:57.777707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:57.837288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f7924-2677-40c0-87ac-8bca450a32a9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:57.945382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:57.946381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:57.946842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:57.947328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:57.948302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:57.948808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:57.950447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:57.951007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbf16-33b7-4a42-ae17-2ada175eb3a7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:57.969774 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 126/262] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:41:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:57.975984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:58.034854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:58.037729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.038202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.038657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.117477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34e011-fc05-4694-aec8-49eac0545ff2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:58.128603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:58.167308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:58.167961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.168232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.168492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] PATH_INFO: `/v3/users/8355c81404214834ab08c5919abf4073` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.170713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] RBAC: Authorizing `identity:get_user(user_id=8355c81404214834ab08c5919abf4073)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:58.172193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:58.181006 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 127/263] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:57 2022] GET /identity/v3/users/8355c81404214834ab08c5919abf4073 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:58.189335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:58.217712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:58.220768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:58.287506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:58.426030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95baaa-7d9e-411d-8422-c64995780e54 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:58.439583 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 137/264] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:58.445592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:58.549638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d2b7b-1601-4d69-a4e6-7605fa6517da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:58.550181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d2b7b-1601-4d69-a4e6-7605fa6517da None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.550511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d2b7b-1601-4d69-a4e6-7605fa6517da None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.550774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d2b7b-1601-4d69-a4e6-7605fa6517da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.633118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551958b0-7f65-4140-9026-3f8e96a757e3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:58.662435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:58.694235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec3af0-10cd-4d73-a9bc-7c334db1dfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:58.695263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec3af0-10cd-4d73-a9bc-7c334db1dfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.696462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec3af0-10cd-4d73-a9bc-7c334db1dfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.696926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec3af0-10cd-4d73-a9bc-7c334db1dfc1 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.700275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec3af0-10cd-4d73-a9bc-7c334db1dfc1 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:58.702639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec3af0-10cd-4d73-a9bc-7c334db1dfc1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:58.708544 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 138/265] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:58 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:41:58.709529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbe3f7-ca9e-4173-8e67-d4044812ef6e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.709813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbe3f7-ca9e-4173-8e67-d4044812ef6e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.710077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbe3f7-ca9e-4173-8e67-d4044812ef6e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.712453 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 139/266] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:58.713290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:58.718024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbe3f7-ca9e-4173-8e67-d4044812ef6e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:58.755022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d2b7b-1601-4d69-a4e6-7605fa6517da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:58.756534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d2b7b-1601-4d69-a4e6-7605fa6517da None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:58.887516 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 128/267] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:41:58.889056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.889336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.889608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.909899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:58.911123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:58.927901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fbe3f7-ca9e-4173-8e67-d4044812ef6e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:58.963787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:58.964505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:58.964775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:58.965041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:58.965771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=8355c81404214834ab08c5919abf4073) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:58.966072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:58.967549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:58.973941 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 140/268] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:58 2022] GET /identity/v3/role_assignments?user.id=8355c81404214834ab08c5919abf4073&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:58.974549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:59.036042 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 129/269] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:59.059644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:59.101278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3197317f-0a14-4795-a1c2-64845ccaa8df None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:59.136922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:59.137662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.137876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.138194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.142368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:59.142368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:59.143210 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 141/270] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:41:58 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:59.149402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:59.170297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:59.296722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2424cd8-c238-49b9-8e2d-c2c00ad5beaa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:59.343670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:59.344692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.345156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.345627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.346544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] RBAC: Adding query filter params (role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=8355c81404214834ab08c5919abf4073) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:59.347047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:59.348718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d11703-f654-4b7b-abe6-5605a60bdb9c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:59.359282 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 130/271] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:41:59 2022] GET /identity/v3/role_assignments?user.id=8355c81404214834ab08c5919abf4073&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=9056804b78cc417c8a175569edc8c78a => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:59.429134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:59.566567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0029a-e2d5-4269-b878-dc8cec7a7e58 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.567632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0029a-e2d5-4269-b878-dc8cec7a7e58 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.568123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0029a-e2d5-4269-b878-dc8cec7a7e58 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.571279 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 131/272] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:59.575211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aedb4f-cc6b-45cf-9936-bfd3d8dec422 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:59.576308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2456-1a73-4ade-8ad7-1ab32779a5ff None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.576774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2456-1a73-4ade-8ad7-1ab32779a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.577265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d2456-1a73-4ade-8ad7-1ab32779a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.615039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ee8d2456-1a73-4ade-8ad7-1ab32779a5ff None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:59.616299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ee8d2456-1a73-4ade-8ad7-1ab32779a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:59.619640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:59.636069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:59.637122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.637599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.638063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.655576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:41:59.662554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:41:59.663558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:41:59.664419 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:41:59.666967 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 142/273] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:41:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:41:59.672984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:59.709574 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 132/274] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:41:59.779109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5190a-168f-4b71-a354-6212a19f4a36 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.779438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5190a-168f-4b71-a354-6212a19f4a36 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.779682 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5190a-168f-4b71-a354-6212a19f4a36 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.784223 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 133/275] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:41:59.791843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda2396-d162-4f31-ae7a-401aadbe7474 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.792683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda2396-d162-4f31-ae7a-401aadbe7474 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.793154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda2396-d162-4f31-ae7a-401aadbe7474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.802948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:41:59.815518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c7af9-e636-459f-b382-d45e348a5e33 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:41:59.819107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fda2396-d162-4f31-ae7a-401aadbe7474 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:59.821467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fda2396-d162-4f31-ae7a-401aadbe7474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:59.861577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:41:59.862825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.864388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.864661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.865339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:41:59.865645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:41:59.867031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebd92b-3fb6-4694-9151-5b0f4047b6ab None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:41:59.886602 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 143/276] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:41:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:41:59.887909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee7bf-536a-4104-9d95-57e82f1ed452 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.888224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee7bf-536a-4104-9d95-57e82f1ed452 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.888535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ee7bf-536a-4104-9d95-57e82f1ed452 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.889756 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 144/277] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:41:59.895370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee7bf-536a-4104-9d95-57e82f1ed452 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:41:59.913855 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 134/278] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:41:59.914917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:41:59.919205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:41:59.919205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:41:59.942797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:41:59.944446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:41:59.951255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.008728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ee7bf-536a-4104-9d95-57e82f1ed452 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.025134 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 135/279] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:00.026036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.051609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.052239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.052568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.052837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] PATH_INFO: `/v3/users/056c163c883245639af749a75aa0e9ea` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.054109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=056c163c883245639af749a75aa0e9ea)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.055908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.058108 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 145/280] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:41:59 2022] GET /identity/v3/users/056c163c883245639af749a75aa0e9ea => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:00.063950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.098836 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.145117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2d97b-10c8-49d4-a106-21fd6d5432dc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.183847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.184499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.184777 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.185434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.197563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:00.203130 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:00.204033 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:00.204825 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:00.205463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d175a-06cc-4387-8994-00d8092ac0c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.205544 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 136/281] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:00.210010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.216964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.238542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.239395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.239825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.240243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.241347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.242791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.249197 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 146/282] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:00.249197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.315713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe30b4-ef0c-4786-af99-e4b6ac2de59e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.337394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.347952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0087ca8-4ed6-400d-a557-0bd334122742 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.349570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.350205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.350475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.350748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.351874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.353333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.354904 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 137/283] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:00.355661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.384767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.385371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.385644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.385909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.386566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:00.386862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.388265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.396180 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 147/284] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:00.396180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.442423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.451788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf516b-58c9-4915-b715-f9d172f6af28 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.484752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.485419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.485666 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.485931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/056c163c883245639af749a75aa0e9ea/roles/253186fb50c242189536f642c4a53561` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.487887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] RBAC: Authorizing `identity:create_grant(project_id=d86b1548e5624d668a2cd70a520c24f5, user_id=056c163c883245639af749a75aa0e9ea, role_id=253186fb50c242189536f642c4a53561)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.489323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.499143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc25d7-0dec-4d18-ae75-d50e210b9d3b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.504625 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 138/285] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:42:00 2022] PUT /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5/users/056c163c883245639af749a75aa0e9ea/roles/253186fb50c242189536f642c4a53561 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:00.505397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.554175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.554822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.555105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.555393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.560880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.571165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:00.576926 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:00.578394 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 148/286] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:00.579221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.626952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4acaa-e2aa-4fb2-bb56-19c632f3750f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.665969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.666894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.667423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.667853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.668664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.671011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.697540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95086ba-c026-4610-a347-83b53f387cd2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.699480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.703300 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 139/287] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:00 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:00.703300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.743612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.744262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.744536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.744803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] PATH_INFO: `/v3/users/27126afaf25342a29d5764a2beea1429` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.760229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] RBAC: Authorizing `identity:get_user(user_id=27126afaf25342a29d5764a2beea1429)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.761735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.763578 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 149/288] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/users/27126afaf25342a29d5764a2beea1429 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:00.769926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.830874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ff0fd-720c-409e-b5a0-23feeeb00d14 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.839513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:00.872868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.873600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.873919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.874192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.874872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:00.875169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.876790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.886259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceea7a6-830e-44fa-a724-6dff48af6dde None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:00.891727 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 140/289] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:00.892941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:00.922392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:00.922966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:00.923265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:00.923538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:00.924265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, user.id=27126afaf25342a29d5764a2beea1429, scope.system=all) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:00.924566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:00.926049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:00.939274 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 150/290] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/role_assignments?user.id=27126afaf25342a29d5764a2beea1429&scope.system=all&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:01.006227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6f2de-7d59-410b-b0f9-c7cf22b3ccee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:01.044823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:01.045716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:01.046163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:01.046567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] PATH_INFO: `/v3/users/d64fc3014247446aa6c088af3ffdc7d8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:01.048031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] RBAC: Authorizing `identity:get_user(user_id=d64fc3014247446aa6c088af3ffdc7d8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:01.049777 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:01.052040 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 141/291] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:00 2022] GET /identity/v3/users/d64fc3014247446aa6c088af3ffdc7d8 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:01.058070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:01.096229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:01.180873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be016e16-319f-4c33-aeea-0bcbc219057a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:01.216800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:01.217375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:01.217647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:01.217912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:01.298353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:01.303937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:01.353825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:01.444839 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 151/292] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:01.450691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:01.524971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:01.618480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106dfb7-41ca-4c14-aa3b-e51b7ad7a3fb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:01.661069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863238a9-0346-45ff-9873-d09a1ca1f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:01.661701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863238a9-0346-45ff-9873-d09a1ca1f478 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:01.661983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863238a9-0346-45ff-9873-d09a1ca1f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:01.662304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863238a9-0346-45ff-9873-d09a1ca1f478 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:01.663474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863238a9-0346-45ff-9873-d09a1ca1f478 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:01.664947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863238a9-0346-45ff-9873-d09a1ca1f478 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:01.666899 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 142/293] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:01 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:01.672883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:01.811973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e1f75-d394-42fe-8f26-470b3dbe5051 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:01.842911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:01.858019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:01.858678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:01.858950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:01.859231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:01.860008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=d64fc3014247446aa6c088af3ffdc7d8) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:01.860317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:01.861762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9b0a6-cb60-47d4-bee3-0858dcf2e239 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:01.871088 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 152/294] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:01 2022] GET /identity/v3/role_assignments?user.id=d64fc3014247446aa6c088af3ffdc7d8&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:02.797381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9e1fe-7d4c-443b-a1a5-de6c6eecfdd2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:02.797963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9e1fe-7d4c-443b-a1a5-de6c6eecfdd2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:02.797963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9e1fe-7d4c-443b-a1a5-de6c6eecfdd2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:02.799543 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 143/295] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:02.804884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:02.805282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:02.805465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:02.824088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:02.825480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:02.895242 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 153/296] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:03.180121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9e1fe-7d4c-443b-a1a5-de6c6eecfdd2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:03.300579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9e1fe-7d4c-443b-a1a5-de6c6eecfdd2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:03.303417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:03.340191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d261b0-5ac0-41df-8494-b6e47c87af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:03.340829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d261b0-5ac0-41df-8494-b6e47c87af1d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.341105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d261b0-5ac0-41df-8494-b6e47c87af1d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.341742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d261b0-5ac0-41df-8494-b6e47c87af1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.354127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d261b0-5ac0-41df-8494-b6e47c87af1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-35d261b0-5ac0-41df-8494-b6e47c87af1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:03.359308 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:03.360240 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:03.361038 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:03.365478 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 144/297] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:03.366847 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:03.371442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0179f-f9b0-4877-8203-6ed6554df4b0 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.371968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0179f-f9b0-4877-8203-6ed6554df4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.372430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e0179f-f9b0-4877-8203-6ed6554df4b0 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.374780 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 145/298] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:03.381635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ca81-e5ea-483d-8648-71fbb4c1f375 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.382601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ca81-e5ea-483d-8648-71fbb4c1f375 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.383609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13ca81-e5ea-483d-8648-71fbb4c1f375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.403217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-af13ca81-e5ea-483d-8648-71fbb4c1f375 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:03.404433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-af13ca81-e5ea-483d-8648-71fbb4c1f375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:03.477028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e0d21-3881-4836-a6e1-e683a5f1f644 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:03.486143 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 146/299] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:03.486143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f356-07a5-4de3-b7ad-5f55b6232944 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.486143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f356-07a5-4de3-b7ad-5f55b6232944 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.486143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036f356-07a5-4de3-b7ad-5f55b6232944 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.486143 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 147/300] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:03.488815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84ea38-a5b1-47dc-938c-928e369e4f4b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.489092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84ea38-a5b1-47dc-938c-928e369e4f4b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.489364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be84ea38-a5b1-47dc-938c-928e369e4f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.500534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:03.515534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-be84ea38-a5b1-47dc-938c-928e369e4f4b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:03.516807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-be84ea38-a5b1-47dc-938c-928e369e4f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:03.518748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:03.519458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.520392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.520392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.520813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:03.521218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:03.522650 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:03.532296 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 154/301] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:03.537984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:03.605850 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 148/302] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:03.607200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d55d6-ea08-4c54-b966-93478e9f146a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.607503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d55d6-ea08-4c54-b966-93478e9f146a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.607767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9d55d6-ea08-4c54-b966-93478e9f146a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.611225 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 149/303] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:03.615250 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.615726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.616199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.636931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:03.638545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:03.646220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9edcb2-2e9d-4e45-a759-ec49b040ae7c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:03.661692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:03.685553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:03.686203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.686489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.686779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] PATH_INFO: `/v3/users/056c163c883245639af749a75aa0e9ea` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.688648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=056c163c883245639af749a75aa0e9ea)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:03.691194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:03.693343 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 155/304] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:03 2022] GET /identity/v3/users/056c163c883245639af749a75aa0e9ea => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:03.700395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:03.727343 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 150/305] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:03.760120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:03.762943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:03.825971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372c4a9-6378-4d26-aad8-153d2dcacbe3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:03.863999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a221e0d-5266-4824-a9f4-921d16aa51ad None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:03.866293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:03.867002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.868130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.868483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.903393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:03.904027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:03.904307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:03.904566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:03.914652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:03.919482 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:03.920369 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:03.921193 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 151/306] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:03.926066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:03.943334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:03.949531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:03.951004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:03.999506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.030237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f8ce8-6853-4ede-9b2e-2dcec17c26b0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.053080 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 156/307] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:42:04.054254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.070745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.071393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.071664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.071936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.082436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:04.087373 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:04.088306 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:04.089108 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 152/308] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:04.089760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.157358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c178a2-d2f2-4398-bc32-79942ab79b2e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.168353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.192118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.192808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.193145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.193475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.193661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb9ee3f-6c8c-44ea-a0c3-1ad4e890703e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.194255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:04.194613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.196066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.204089 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 157/309] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.208938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.216860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.228934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.229559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.229840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.230104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.231237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.232700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.234252 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 153/310] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.235004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.306288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd762f-083b-409d-9b4e-7239fb7c6ed4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.333691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e87043-f5ef-47d9-a89c-4ef7bc041e0e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.339150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.339800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.340059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.340324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.341450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.342876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.344546 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 158/311] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.345288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.370492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.371100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.371397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.371665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.372329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:04.372644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.374029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.383463 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 154/312] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.384075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.387425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.427321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.450358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c198f1d-965a-4ff2-a557-e665b1ccc5cf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.484461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.484631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2af32-a6dd-4fd8-9bbb-d76c32472f90 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.485090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.485370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.485668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] PATH_INFO: `/v3/users/27126afaf25342a29d5764a2beea1429` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.486822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] RBAC: Authorizing `identity:get_user(user_id=27126afaf25342a29d5764a2beea1429)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.488400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.489993 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 159/313] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/users/27126afaf25342a29d5764a2beea1429 => generated 293 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.490743 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.521487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.522088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.522358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.522620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] PATH_INFO: `/v3/groups` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.523277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.524712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.542445 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 155/314] 10.222.0.23 () {60 vars in 1134 bytes} [Sun Sep 11 00:42:04 2022] POST /identity/v3/groups => generated 229 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:04.551704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.606631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f81d7-26ef-4844-8ec8-4f1c1374d7fb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.614530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.646324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.647067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.647431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.647704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.648441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] RBAC: Adding query filter params (role.id=253186fb50c242189536f642c4a53561, scope.project.id=d86b1548e5624d668a2cd70a520c24f5, user.id=056c163c883245639af749a75aa0e9ea) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:04.648744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.650257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.656539 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 160/315] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/role_assignments?user.id=056c163c883245639af749a75aa0e9ea&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=253186fb50c242189536f642c4a53561 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.657361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.669547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168dcca-0e4c-4857-924a-905766931aae None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.680525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.722058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.722836 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.723213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.723555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] PATH_INFO: `/v3/users/aadb19c1edba449397c4ae1c99becb4e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.742987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] RBAC: Authorizing `identity:get_user(user_id=aadb19c1edba449397c4ae1c99becb4e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.744470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.746439 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 156/316] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/users/aadb19c1edba449397c4ae1c99becb4e => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:04.753860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:04.807644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64758fc6-b69f-4930-8a91-5b2d141849c4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.846231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908b81a-939e-403f-a17d-b16b33215ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.847723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908b81a-939e-403f-a17d-b16b33215ea3 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.848139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908b81a-939e-403f-a17d-b16b33215ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.848549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908b81a-939e-403f-a17d-b16b33215ea3 None admin] PATH_INFO: `/v3/system/users/27126afaf25342a29d5764a2beea1429/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.850237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908b81a-939e-403f-a17d-b16b33215ea3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=27126afaf25342a29d5764a2beea1429, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.851871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908b81a-939e-403f-a17d-b16b33215ea3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.864057 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 161/317] 10.222.0.23 () {58 vars in 1334 bytes} [Sun Sep 11 00:42:04 2022] PUT /identity/v3/system/users/27126afaf25342a29d5764a2beea1429/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:04.885871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b20742-86af-45eb-baba-f325c33ab8b6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:04.915267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:04.928853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:04.929476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:04.929759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:04.930032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:04.930776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] RBAC: Adding query filter params (role.id=253186fb50c242189536f642c4a53561, user.id=aadb19c1edba449397c4ae1c99becb4e, scope.system=all) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:04.931076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:04.932607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d027368-7948-475f-902f-4a405848aab4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:04.939451 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 157/318] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Sep 11 00:42:04 2022] GET /identity/v3/role_assignments?user.id=aadb19c1edba449397c4ae1c99becb4e&scope.system=all&role.id=253186fb50c242189536f642c4a53561 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:06.766514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d2873-dad5-4b32-8455-da022f9af9b8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:06.767144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d2873-dad5-4b32-8455-da022f9af9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:06.767144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d2873-dad5-4b32-8455-da022f9af9b8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:06.768604 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 162/319] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:06.773776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea15e-c691-4133-bfb7-6fe40b8f26ae None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:06.774193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea15e-c691-4133-bfb7-6fe40b8f26ae None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:06.774266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ea15e-c691-4133-bfb7-6fe40b8f26ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:06.793142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cb3ea15e-c691-4133-bfb7-6fe40b8f26ae None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:06.794335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb3ea15e-c691-4133-bfb7-6fe40b8f26ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:06.866227 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 158/320] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:07.069530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01098e2-9c9a-4008-ba2a-aa89321d3500 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.069709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01098e2-9c9a-4008-ba2a-aa89321d3500 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.070018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01098e2-9c9a-4008-ba2a-aa89321d3500 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.071499 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 163/321] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:07.073465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102ef50-c6df-488d-be68-1b5ae786a7de None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.073750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102ef50-c6df-488d-be68-1b5ae786a7de None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.073990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102ef50-c6df-488d-be68-1b5ae786a7de None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.075522 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 159/322] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:07.076198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.076461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.076738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.080074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.080427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.080617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.095118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:07.096338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:07.098416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:07.099577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:07.168965 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 164/323] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:07.169825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.173687 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 160/324] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:07.176425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:07.273379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211dc84-5c51-4a35-9e7e-8fa7d00f40ac None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.306749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.307410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.307682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.307948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.309054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:07.310845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:07.312631 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 165/325] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:07.318071 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.364883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:07.419259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75772-8206-4843-8a4d-e9b8eefbc7e4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.434239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.453209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.453819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.454119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.454387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] PATH_INFO: `/v3/groups` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.455015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:07.456641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:07.472860 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 161/326] 10.222.0.23 () {60 vars in 1134 bytes} [Sun Sep 11 00:42:07 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:07.473500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.519007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:07.554469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf126c-b5be-46ea-9ced-8c6bb63c638a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.595300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.595943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.596212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.596475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.596569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8729b-ab7e-45f9-a599-31357f3ba806 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.605515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:07.608080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:07.613316 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:07.614299 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:07.615167 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 166/327] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:07.621703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.639951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.640593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.640865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.641127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.652111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:07.657051 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:07.657931 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:07.658777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:07.659414 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 162/328] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:07.664813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.735081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fc498-cd2c-4915-9822-5033371bf72f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.769422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:07.776464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.777419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.777701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.778111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.778931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:07.779279 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:07.781140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:07.787732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61061e6-dc3f-4ca6-86db-ca15f1eb6c35 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.792836 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 167/329] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:07.798613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.830854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.831521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.831797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.832063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.832732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:07.833029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:07.834423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:07.844945 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 163/330] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:07.853074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.897563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:07.918167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc0c58-9aaa-4e60-849d-d863f760fc4a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:07.956185 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:07.956900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:07.957185 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:07.957436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] PATH_INFO: `/v3/users/aadb19c1edba449397c4ae1c99becb4e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:07.959082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] RBAC: Authorizing `identity:get_user(user_id=aadb19c1edba449397c4ae1c99becb4e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:07.960275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:07.962299 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 168/331] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/users/aadb19c1edba449397c4ae1c99becb4e => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:07.969574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:07.989366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d6b75-d9d0-434b-91e7-9c9b3a0fcf84 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:08.028142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:08.029123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:08.029585 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:08.030154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] PATH_INFO: `/v3/users/27126afaf25342a29d5764a2beea1429` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:08.031563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=27126afaf25342a29d5764a2beea1429)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:08.033495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:08.036576 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 164/332] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:07 2022] GET /identity/v3/users/27126afaf25342a29d5764a2beea1429 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:08.039474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:08.042319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:08.086837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103044e6-ad49-44b7-ad9a-5a7da0219c80 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:08.127563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d95cda-ad2c-41a6-94f1-8f8bd16deeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:08.128247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d95cda-ad2c-41a6-94f1-8f8bd16deeac None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:08.128525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d95cda-ad2c-41a6-94f1-8f8bd16deeac None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:08.128797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d95cda-ad2c-41a6-94f1-8f8bd16deeac None admin] PATH_INFO: `/v3/system/users/aadb19c1edba449397c4ae1c99becb4e/roles/253186fb50c242189536f642c4a53561` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:08.130371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d95cda-ad2c-41a6-94f1-8f8bd16deeac None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=aadb19c1edba449397c4ae1c99becb4e, role_id=253186fb50c242189536f642c4a53561)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:08.132291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d95cda-ad2c-41a6-94f1-8f8bd16deeac None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:08.154401 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 169/333] 10.222.0.23 () {58 vars in 1334 bytes} [Sun Sep 11 00:42:07 2022] PUT /identity/v3/system/users/aadb19c1edba449397c4ae1c99becb4e/roles/253186fb50c242189536f642c4a53561 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:42:08.187632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:08.195472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a4da94-3d19-40b2-9ff8-7f942bb05cc3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:08.233901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:08.234526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:08.234804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:08.235069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:08.235837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] RBAC: Adding query filter params (role.id=1610cc07f3b24392a709fa07d663cb77, user.id=27126afaf25342a29d5764a2beea1429, scope.system=all) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:08.236188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:08.237842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1628b-d616-4150-a143-14ef6db6427d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:08.251740 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 165/334] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Sep 11 00:42:08 2022] GET /identity/v3/role_assignments?user.id=27126afaf25342a29d5764a2beea1429&scope.system=all&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:09.874640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e7489-63a6-43ae-8c4b-e48b17c6e64d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:09.875220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e7489-63a6-43ae-8c4b-e48b17c6e64d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:09.875220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e7489-63a6-43ae-8c4b-e48b17c6e64d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:09.876782 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 170/335] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:09.882986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025464c9-b036-48c9-bfdf-ea282292f343 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:09.883425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025464c9-b036-48c9-bfdf-ea282292f343 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:09.883629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025464c9-b036-48c9-bfdf-ea282292f343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:09.902140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-025464c9-b036-48c9-bfdf-ea282292f343 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:09.903368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-025464c9-b036-48c9-bfdf-ea282292f343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:09.973897 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 166/336] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:10.255232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e7489-63a6-43ae-8c4b-e48b17c6e64d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:10.394908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e7489-63a6-43ae-8c4b-e48b17c6e64d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:10.425065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:10.433624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:10.434261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.434534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.435163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.441185 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d17407-6a34-403c-b5e8-900f4b0b97b9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.441463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d17407-6a34-403c-b5e8-900f4b0b97b9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.441742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d17407-6a34-403c-b5e8-900f4b0b97b9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.443089 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 167/337] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:10.449225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.449744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.450203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:10.459213 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:10.460084 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:10.460944 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 171/338] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:10.466352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:10.485206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:10.486863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:10.597347 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 168/339] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:10.624254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe35b8-df90-4296-813c-f63b5a6be5dc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:10.663315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:10.663933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.664206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.664473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.665157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:10.665454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:10.666868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1be41-6210-4ff0-b1da-7e3b13f0318c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:10.679370 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 172/340] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:10.685476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:10.701451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:10.795966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4185c55-da1e-4d7c-8e31-74f313b96e25 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:10.839620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:10.840240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.840510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.840772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.903686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec9c9-ec11-445c-ae5f-2fc139ced683 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.904199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec9c9-ec11-445c-ae5f-2fc139ced683 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.904671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ec9c9-ec11-445c-ae5f-2fc139ced683 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.906617 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 173/341] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:10.912353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:10.912900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:10.913170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:10.928296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:10.929819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:10.944668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:10.946387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:10.960609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:11.031574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:11.085452 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 174/342] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:11.086370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:11.170414 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 169/343] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:42:11.177863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:11.281015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9fc26-5983-41c9-80d1-0c65cb878c36 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:11.344254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1ad8d-2731-4280-8db0-112ef3c10374 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:11.356744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:11.357412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.357677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.357936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.365204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:11.382912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:11.392335 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:11.393242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:11.394150 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 175/344] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:11.397896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:11.397896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.397896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.397896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.398518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:11.399828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:11.407572 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 170/345] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:11 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:11.408530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:11.411794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:11.476690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:11.546315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b4583-bf21-45c8-9fdd-becbadbfa94e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:11.587569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:11.588181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.588451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.588716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.589379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:11.589668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:11.593896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dec28-7ec0-43ce-9692-86e6ad446947 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:11.605220 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 176/346] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:11.606736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295d266-1b49-491b-a248-00c3b3d1f778 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.607014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295d266-1b49-491b-a248-00c3b3d1f778 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.607303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5295d266-1b49-491b-a248-00c3b3d1f778 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.608766 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 177/347] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:11.613320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295d266-1b49-491b-a248-00c3b3d1f778 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:11.634388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09d8c0-23ce-4322-9cc2-d7cee2c9bdd5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:11.676478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058f3425-b85d-4a64-b90b-423aae01cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:11.677139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3425-b85d-4a64-b90b-423aae01cb95 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.677415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3425-b85d-4a64-b90b-423aae01cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.677684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058f3425-b85d-4a64-b90b-423aae01cb95 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.684132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f3425-b85d-4a64-b90b-423aae01cb95 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:11.685981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058f3425-b85d-4a64-b90b-423aae01cb95 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:11.692084 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 171/348] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:11 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:11.693335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.693616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.693885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.719445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:11.720670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:11.732322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:11.739972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5295d266-1b49-491b-a248-00c3b3d1f778 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:11.792991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:11.793926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.794351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.794752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.803649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:11.831782 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 172/349] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:11.832700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:11.927337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:11.928825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:11.946124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd97e6a-19ed-47e4-a0b9-f3cb182d939b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:11.987275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:11.988181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:11.988598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:11.989008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:11.990463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:11.991054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:11.992958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:12.000148 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 173/350] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:11 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:12.001193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.050414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:12.057951 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 178/351] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:12.063613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:12.064016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.209347 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fa339-bba2-4e45-a19a-13de9cfd128a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.241380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad41973-fe33-46c7-996b-d44166c7bb2c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.288175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.288828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.289178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.289460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.290601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:12.290705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4db35b-7af4-41fa-bbdf-7a3ca4edc102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.291283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4db35b-7af4-41fa-bbdf-7a3ca4edc102 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.291560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4db35b-7af4-41fa-bbdf-7a3ca4edc102 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.291823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4db35b-7af4-41fa-bbdf-7a3ca4edc102 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.292168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:12.294496 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 179/352] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:12.304077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.319681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4db35b-7af4-41fa-bbdf-7a3ca4edc102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8b4db35b-7af4-41fa-bbdf-7a3ca4edc102 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:12.325309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:12.326243 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 11 00:42:12.327058 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:12.334245 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 174/353] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:12.335921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788d78-08c1-4fa6-8f77-7f4e3fdd4b32 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.336418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788d78-08c1-4fa6-8f77-7f4e3fdd4b32 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.336868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0788d78-08c1-4fa6-8f77-7f4e3fdd4b32 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.339904 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 175/354] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:12.345071 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.345573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.346099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.376135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:12.377338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:12.393355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:12.433853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aabbcd-78f2-4317-84ac-dcd45c87c2a9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.462687 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 176/355] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:12.463957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.472918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.473800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.474245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.474717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.476131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:12.478125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:12.480266 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 180/356] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:12.481287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.499599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:12.587887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c3dd3-5b7e-4e6b-92b0-092baeff9e31 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.624128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08769bac-7cb0-4adb-a8c6-1d31c69e9c51 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.648686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.649661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.650154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.651792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.652668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:12.653180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:12.656070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:12.673229 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 177/357] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:12.675195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.694620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.695753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.697670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.698170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.703517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:12.717216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:12.725875 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:12.726862 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:12.727743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:12.730723 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 181/358] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:12.734525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.815630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:12.821040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f4f83-41bd-4a4c-92b7-2ee06f3c7aa8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.870687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.871254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.871536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.871805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.872542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:12.872902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:12.874409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:12.880668 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 178/359] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=9056804b78cc417c8a175569edc8c78a => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:12.881676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:12.906348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8010bfe-23cb-441f-bf45-9003fe292cfb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:12.957338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:12.957988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:12.958378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:12.958647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] PATH_INFO: `/v3/users/aadb19c1edba449397c4ae1c99becb4e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:12.960763 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] RBAC: Authorizing `identity:get_user(user_id=aadb19c1edba449397c4ae1c99becb4e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:12.962685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:12.965819 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 182/360] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/users/aadb19c1edba449397c4ae1c99becb4e => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:12.968045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.036988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b2123-eed0-4414-a643-444e1ca56bc6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.056516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:13.084298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.084844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.085116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.085377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.086358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] RBAC: Adding query filter params (name=member) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:13.086469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:13.087854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:13.099256 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 179/361] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:13.100166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.104952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782bb5c0-d45b-4f6c-afdb-ecd7730581ae None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.124450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:13.149566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.150629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.151134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.151643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.168006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:13.173870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:13.174876 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:13.177215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:13.179507 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 183/362] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:13.180587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.229138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6895f-cafc-4e90-98c7-bf234bf37c3e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.281266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.281880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.282198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.282466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.283726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] RBAC: Adding query filter params (role.id=253186fb50c242189536f642c4a53561, user.id=aadb19c1edba449397c4ae1c99becb4e, scope.system=all) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:13.284035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:13.285960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:13.296695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b944e-a123-4d40-8cbf-1338724ab9ce None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.308100 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 180/363] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Sep 11 00:42:13 2022] GET /identity/v3/role_assignments?user.id=aadb19c1edba449397c4ae1c99becb4e&scope.system=all&role.id=253186fb50c242189536f642c4a53561 => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:13.309151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.346771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:13.381282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.381818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.382086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.382355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.415095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:13.460893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af949b-8f8e-43f9-a168-92e2efe06b51 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.535933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:13.537538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:13.546706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.547734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.548208 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.548687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.549520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:13.549822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:13.551302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5bd1a-e9f8-44b5-a18b-0263b504f277 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:13.562183 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 181/364] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:13.563854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782adaae-0bd5-46e4-94fb-c0ff302d915a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.564331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782adaae-0bd5-46e4-94fb-c0ff302d915a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.565937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782adaae-0bd5-46e4-94fb-c0ff302d915a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.567459 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 182/365] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:13.574863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.575543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.576047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.600919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:13.602455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:13.671001 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 184/366] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:13.671855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.680535 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 183/367] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:13.681676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.722015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:13.817497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:13.868204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b530204b-130a-4c6d-80a4-79f184946ed2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.871797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b0e23-bf10-4f36-a71e-207fbf3d155c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:13.916295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.916961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.917231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.917497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:13.919070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:13.920977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:13.923301 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 184/368] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:13 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:13.928491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:13.929579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:13.932057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:13.932322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:13.932581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.052464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.054044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.091622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.110802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e339f1-dabc-4690-a058-81d1152306a3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.147861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.148512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.148788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.149058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.150234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.151749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.153775 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 185/369] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:13 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:14.154615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.158746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.171338 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 185/370] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:14.176230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.265585 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee3ed7-8a67-4c1f-8905-e34314d4173d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.281856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d8f7f-2196-4900-b233-0cb96c00df27 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.301186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.301891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.302167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.302453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.306298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.315724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:14.318166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.318835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.319205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.319541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.320372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=d86b1548e5624d668a2cd70a520c24f5) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:14.320736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:14.321069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:14.321934 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:14.322833 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 186/371] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:14.323526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.323593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.329250 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 186/372] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:14.330311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.395596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.447403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f01456-a148-4c42-8431-01d7c4e151f3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.453208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6db32-d9dc-4939-8868-676c69a33a89 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.491158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.492035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.492403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.492796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.495095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.496811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.499120 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 187/373] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:14.499326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.499624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.499906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.500171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.500856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:14.501158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.502621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.505332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.513920 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 187/374] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:14.519898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.591090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.638325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf70ef-7e9a-4e06-92c6-2aa660ecc226 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.650588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b13c7f-23d3-4461-a91e-498bc558de7f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.686860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.687556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.687834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.688104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.689870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.691528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.695296 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 188/375] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:14.695551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.697852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.697852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.697852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.697852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.700982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:14.824316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.825976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.901811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:14.906173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf84557-4d38-40be-8b04-345335ba05a7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:14.951417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:14.951978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.952249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.952512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.953382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=d86b1548e5624d668a2cd70a520c24f5) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:14.953697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:14.955983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9bccc-f71e-4abe-8269-c17852ed0457 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:14.965485 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 189/376] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=9056804b78cc417c8a175569edc8c78a => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:14.967692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9018e1-220e-444e-8249-43a33904bbaa None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.969498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9018e1-220e-444e-8249-43a33904bbaa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.970713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9018e1-220e-444e-8249-43a33904bbaa None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.973085 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 190/377] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:14.976373 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 188/378] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:14.980813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:14.981456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:14.981936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:14.988635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:15.021707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:15.026985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:15.151899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:15.174292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fb995-af00-4997-b00f-9624ae1b539c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:15.180608 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 191/379] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:15.225729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:15.226284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.226557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.226816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] PATH_INFO: `/v3/projects/857538f273814e30bf74c0248d309754` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.229673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] RBAC: Authorizing `identity:get_project(project_id=857538f273814e30bf74c0248d309754)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:15.231117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:15.233233 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 189/380] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:14 2022] GET /identity/v3/projects/857538f273814e30bf74c0248d309754 => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:15.238686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:15.363219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05d90e-a339-49ff-9bb3-0170eee3ba56 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:15.400413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:15.401057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.401329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.401596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] PATH_INFO: `/v3/groups/8ef3c4b63f8746b697af4df200888711` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.406668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] RBAC: Authorizing `identity:get_group(group_id=8ef3c4b63f8746b697af4df200888711)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:15.408154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:15.410214 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 192/381] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:15 2022] GET /identity/v3/groups/8ef3c4b63f8746b697af4df200888711 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:15.416034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:15.447903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:15.496540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:15.549647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac2393-d429-4041-9d7e-a09c67aed46d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:15.558952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:15.590676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:15.591312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.591681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.591945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.592669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] RBAC: Adding query filter params (group.id=8ef3c4b63f8746b697af4df200888711, role.id=77132358e0b64de4a63750b81e359340, scope.project.id=857538f273814e30bf74c0248d309754) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:15.592966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:15.594561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:15.601177 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 190/382] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:15 2022] GET /identity/v3/role_assignments?group.id=8ef3c4b63f8746b697af4df200888711&scope.project.id=857538f273814e30bf74c0248d309754&role.id=77132358e0b64de4a63750b81e359340 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:15.603737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07fcb3-34ed-4d8b-afd8-d0783fcccff9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:15.642895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9d9a9-e776-456a-98c5-56ccc6eaef38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:15.643590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9d9a9-e776-456a-98c5-56ccc6eaef38 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.644094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9d9a9-e776-456a-98c5-56ccc6eaef38 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.644360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9d9a9-e776-456a-98c5-56ccc6eaef38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.657184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9d9a9-e776-456a-98c5-56ccc6eaef38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-63f9d9a9-e776-456a-98c5-56ccc6eaef38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:15.662757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:15.664869 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:15.666839 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 193/383] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:15.670396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:15.781116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:15.803570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c94f9f-4cf0-4511-b9a1-12d2017f0cef None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.803891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c94f9f-4cf0-4511-b9a1-12d2017f0cef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.804060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63e69ff-1512-4d34-9fbe-6c2cd6ae07a6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:15.804240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c94f9f-4cf0-4511-b9a1-12d2017f0cef None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.805812 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 194/384] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:15.811948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.812255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.812480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.833238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:15.834471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:15.864933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:15.865554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:15.865826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:15.866143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:15.866774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:15.867077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:15.868485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:15.886662 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 191/385] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:15.899577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:15.911234 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 195/386] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:16.013900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a476fc-c80f-471f-9919-1bba4a1b87e4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:16.048282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:16.048835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:16.049111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:16.049361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:16.050521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:16.052112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:16.054071 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 192/387] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:15 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:16.060049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:16.113436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:16.160942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb3eff-8e86-4088-afe1-2acac9e45f6b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:16.195635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:16.196331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:16.196605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:16.196884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:16.198150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:16.199649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:16.201297 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 196/388] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:16 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:16.207210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:16.224012 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:16.239696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:16.375263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4777925-fd2f-497a-8a51-bf4975aab5d0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:16.398349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e06a1-4077-4dea-a4c8-66d3ff6bf1f9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:16.426273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:16.438630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:16.439670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:16.440147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:16.440673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:16.453396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:16.459647 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:16.460534 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:16.461355 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:16.465450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:16.465450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:16.465768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:16.465988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/groups/365252aeff4640feb74434cf3acf1f66/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:16.466654 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 197/389] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:16.470307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] RBAC: Authorizing `identity:create_grant(project_id=01418d2a80e444dfa5e16569dd4fed3f, group_id=365252aeff4640feb74434cf3acf1f66, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:16.470307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:16.473207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:16.494474 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 193/390] 10.222.0.23 () {58 vars in 1442 bytes} [Sun Sep 11 00:42:16 2022] PUT /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/groups/365252aeff4640feb74434cf3acf1f66/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:16.596962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22bb3c-719a-4928-a54e-10a30e822448 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:16.631739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:16.632725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:16.633203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:16.633668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:16.634535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:16.635031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:16.636629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:16.646619 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 198/391] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:16.652729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:16.736395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:16.787029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae91ff-fad4-4caf-b40b-9910f8709581 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:16.830406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:16.831044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:16.831366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:16.831633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:16.832721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:16.834155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:16.836824 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 194/392] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:16 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:16.841588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:16.905210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:16.986276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8e810-c41f-453c-adf4-4200b0157d71 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:17.050677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f46b-abc0-403d-99d7-5df6dce843fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:17.051437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f46b-abc0-403d-99d7-5df6dce843fa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:17.051714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f46b-abc0-403d-99d7-5df6dce843fa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:17.051981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f46b-abc0-403d-99d7-5df6dce843fa None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:17.053190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f46b-abc0-403d-99d7-5df6dce843fa None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:17.054637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f46b-abc0-403d-99d7-5df6dce843fa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:17.056636 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 199/393] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:16 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:17.062946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:17.218139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e9b9c-3603-4a7f-b8dd-fdc2085f43de None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:17.253926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739ed66-2ece-4933-823c-8817c973fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:17.254592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ed66-2ece-4933-823c-8817c973fee8 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:17.254863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ed66-2ece-4933-823c-8817c973fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:17.255122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739ed66-2ece-4933-823c-8817c973fee8 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/groups/365252aeff4640feb74434cf3acf1f66/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:17.257245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ed66-2ece-4933-823c-8817c973fee8 None admin] RBAC: Authorizing `identity:create_grant(project_id=01418d2a80e444dfa5e16569dd4fed3f, group_id=365252aeff4640feb74434cf3acf1f66, role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:17.258679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739ed66-2ece-4933-823c-8817c973fee8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:17.273097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:17.277340 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 195/394] 10.222.0.23 () {58 vars in 1442 bytes} [Sun Sep 11 00:42:17 2022] PUT /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f/groups/365252aeff4640feb74434cf3acf1f66/roles/9056804b78cc417c8a175569edc8c78a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:17.668106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae4dc3-8977-4204-b90e-acd7545f3822 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:17.669273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae4dc3-8977-4204-b90e-acd7545f3822 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:17.669772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae4dc3-8977-4204-b90e-acd7545f3822 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:17.671769 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 200/395] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:17.679604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce70dd-1719-4a3a-b921-bd206eb1bca6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:17.679903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce70dd-1719-4a3a-b921-bd206eb1bca6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:17.680143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce70dd-1719-4a3a-b921-bd206eb1bca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:17.700854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b5ce70dd-1719-4a3a-b921-bd206eb1bca6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:17.702079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b5ce70dd-1719-4a3a-b921-bd206eb1bca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:17.785165 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 196/396] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:17.856494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b58e-bdef-4d8a-b1b3-091fd99805a1 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:17.856803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b58e-bdef-4d8a-b1b3-091fd99805a1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:17.857039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0b58e-bdef-4d8a-b1b3-091fd99805a1 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:17.858483 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 201/397] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:17.873050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:17.873331 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:17.873599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:17.896240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:17.897422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:17.979464 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 197/398] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:18.101860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b58e-bdef-4d8a-b1b3-091fd99805a1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:18.236085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0b58e-bdef-4d8a-b1b3-091fd99805a1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:18.247995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:18.281801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:18.282438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.282710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.282976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.300119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:18.304944 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:18.305831 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:18.306649 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 202/399] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:18.314171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:18.358801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:18.399597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:18.467734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fb4a3-661d-4af1-8af2-cb5cb34d9e93 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:18.535218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:18.535761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.536028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.536287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.536962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:18.537254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:18.538635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:18.561027 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 198/400] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:18.565277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:18.584831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72307a67-f3bf-452e-8f3e-d9f68fab63c8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:18.663767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:18.664415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.664686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.664949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.691824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:18.697505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:18.698417 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:18.699406 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 203/401] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:18.712310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:18.768087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:18.770481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eaaa9-58f4-4ec4-ba97-75b30f8b1d6c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:18.806985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:18.821469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b2bc3-fb5a-4923-adef-9459db1d08bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:18.822245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b2bc3-fb5a-4923-adef-9459db1d08bb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.822613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b2bc3-fb5a-4923-adef-9459db1d08bb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.823015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b2bc3-fb5a-4923-adef-9459db1d08bb None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.825519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b2bc3-fb5a-4923-adef-9459db1d08bb None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:18.827063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b2bc3-fb5a-4923-adef-9459db1d08bb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:18.829128 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 199/402] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:18 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:18.835003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd949eb-fedf-46f3-8ada-5290b02d3d27 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.835679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd949eb-fedf-46f3-8ada-5290b02d3d27 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.835679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd949eb-fedf-46f3-8ada-5290b02d3d27 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.837412 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 200/403] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:18.838308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd949eb-fedf-46f3-8ada-5290b02d3d27 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:18.860017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c562c-48e8-438e-b024-21b94c29d48f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:18.900931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:18.902073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.902971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.903444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.904530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:18.905030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:18.906742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462fa8a-3d89-4489-8308-008fe876caff None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:18.916397 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 204/404] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:18.917854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:18.922726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:18.922726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:18.943748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:18.944951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:18.981624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd949eb-fedf-46f3-8ada-5290b02d3d27 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.024084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.025055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.025488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.025914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.027206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:19.028878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:19.030727 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 201/406] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:18 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:19.030833 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 205/406] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:19.037135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.037281 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.075031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:19.129893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:19.215582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54c83f-f9a0-48c4-b5aa-bfb55eeae48f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.234617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b05043-5afa-415c-bf75-3c2b491f3d4a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.283409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e91b10-0c67-47b9-a84d-6455cbee42db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.284138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91b10-0c67-47b9-a84d-6455cbee42db None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.284487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91b10-0c67-47b9-a84d-6455cbee42db None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.284731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e91b10-0c67-47b9-a84d-6455cbee42db None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5/groups/365252aeff4640feb74434cf3acf1f66/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.287272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e91b10-0c67-47b9-a84d-6455cbee42db None admin] RBAC: Authorizing `identity:create_grant(project_id=d86b1548e5624d668a2cd70a520c24f5, group_id=365252aeff4640feb74434cf3acf1f66, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:19.288720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e91b10-0c67-47b9-a84d-6455cbee42db None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:19.290552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.291227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.291633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.291906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.293043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:19.294469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:19.296397 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 206/407] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:19 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:19.302216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.329570 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 202/408] 10.222.0.23 () {58 vars in 1442 bytes} [Sun Sep 11 00:42:19 2022] PUT /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5/groups/365252aeff4640feb74434cf3acf1f66/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:42:19.344028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0545-0629-4e6b-977b-cbe21da96118 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.344354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0545-0629-4e6b-977b-cbe21da96118 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.344572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de0545-0629-4e6b-977b-cbe21da96118 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.346043 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 203/409] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:19.360731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.361062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.361352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.392390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:19.393648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:19.399440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:19.439487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aee440-59e2-409b-9ac3-6e019d2ccf62 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.487217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.488233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.490323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.490851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.492359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:19.492766 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 204/410] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:19.493685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.494463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:19.496775 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 207/411] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:19 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:19.504057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.590053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:19.631620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e209d-bff5-4f3f-b383-ce830df73ba4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.662731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d43b40-ace5-4488-8d24-206e1fac372c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.672618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.673283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.673555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.674189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5/groups/365252aeff4640feb74434cf3acf1f66/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.677142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] RBAC: Authorizing `identity:create_grant(project_id=d86b1548e5624d668a2cd70a520c24f5, group_id=365252aeff4640feb74434cf3acf1f66, role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:19.678650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:19.695477 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 208/412] 10.222.0.23 () {58 vars in 1442 bytes} [Sun Sep 11 00:42:19 2022] PUT /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5/groups/365252aeff4640feb74434cf3acf1f66/roles/9056804b78cc417c8a175569edc8c78a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:19.705127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.705825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.706139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.706492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.719101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:19.724434 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:19.725611 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:19.726532 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 205/413] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:19.732170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.754532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:19.838136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.852079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe2a0-ffec-4c3b-9be0-f63e282bef0c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.895636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:19.896256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:19.896583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:19.896856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:19.898086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:19.898532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:19.900358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:19.912039 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 209/414] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:19.917779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:19.953897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4e26-2053-47a0-ad90-9cc88da4169b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:19.965087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.007638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.009384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.010374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.010875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.014559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.033639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f291a-71bf-41c8-96dc-44eb19edfb14 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.040117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:20.045176 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:20.046078 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:20.047287 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 206/415] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:20.052440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.084377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.085979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.086465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.086950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] PATH_INFO: `/v3/groups/8ef3c4b63f8746b697af4df200888711` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.088202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] RBAC: Authorizing `identity:get_group(group_id=8ef3c4b63f8746b697af4df200888711)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:20.089737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:20.095046 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 210/416] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:19 2022] GET /identity/v3/groups/8ef3c4b63f8746b697af4df200888711 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:20.097589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.182354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb6ca25-c5ac-4902-9896-51e5dc64ef48 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.230128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.230760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.231031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.231320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.232200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:20.232371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:20.233808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ecc41-1111-45c8-978f-f06b5c636533 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:20.241927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8f846-6840-4cb4-b25e-85543e2ed6f1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.253296 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 207/417] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:20.254920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0453f09-adca-4989-a661-9d365776933d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.255436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0453f09-adca-4989-a661-9d365776933d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.255918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0453f09-adca-4989-a661-9d365776933d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.258012 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 208/418] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:20.263594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0453f09-adca-4989-a661-9d365776933d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.291765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56a626-c0f1-47fb-a7f1-4fc24975ec2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.292388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56a626-c0f1-47fb-a7f1-4fc24975ec2f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.292665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56a626-c0f1-47fb-a7f1-4fc24975ec2f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.292953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56a626-c0f1-47fb-a7f1-4fc24975ec2f None admin] PATH_INFO: `/v3/projects/857538f273814e30bf74c0248d309754` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.294064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56a626-c0f1-47fb-a7f1-4fc24975ec2f None admin] RBAC: Authorizing `identity:get_project(project_id=857538f273814e30bf74c0248d309754)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:20.296287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56a626-c0f1-47fb-a7f1-4fc24975ec2f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:20.303107 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 211/419] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:20 2022] GET /identity/v3/projects/857538f273814e30bf74c0248d309754 => generated 344 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:20.303107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.303107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.303107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.307077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.322972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:20.324361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:20.334311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.403646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0453f09-adca-4989-a661-9d365776933d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.408638 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 212/420] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:20.409890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.503973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.504542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.504827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.505097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.514764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382a543-c07b-45e2-85c0-92b31b1a4f08 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.550810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.551854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.552319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.552800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] PATH_INFO: `/v3/projects/857538f273814e30bf74c0248d309754/groups/8ef3c4b63f8746b697af4df200888711/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.554870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] RBAC: Authorizing `identity:create_grant(project_id=857538f273814e30bf74c0248d309754, group_id=8ef3c4b63f8746b697af4df200888711, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:20.556574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:20.576917 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 213/421] 10.222.0.23 () {58 vars in 1442 bytes} [Sun Sep 11 00:42:20 2022] PUT /identity/v3/projects/857538f273814e30bf74c0248d309754/groups/8ef3c4b63f8746b697af4df200888711/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:20.628796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:20.630669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:20.672838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.707608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.712029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.778189 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 209/422] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:20.784239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.841377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91124ef-4a51-4452-901a-b70b1d41abe1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.903975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.904881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.905317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.905720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.920441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:20.926896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:20.927993 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:20.929008 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:20.931947 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 214/423] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:20.938210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:20.950464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24036201-0d05-45df-bc23-7ab0504cdadb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:20.982284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:20.993016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:20.993791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:20.994358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:20.995399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:20.998015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:21.000115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:21.005012 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 210/424] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:20 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:21.012646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:21.036555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:21.109887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845525c-fb87-49bf-bf88-e3db6f5d6776 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:21.187313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:21.188050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:21.188449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:21.188814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:21.189587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:21.189986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:21.191605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:21.209256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd04908-fef5-4f0b-b1af-302a80faa4f0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:21.211137 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 215/425] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:21.216251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:21.261968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:21.262776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:21.263290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:21.263655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:21.268607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:21.271636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:21.273684 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 211/426] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:21 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:21.280905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:21.390653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:21.413934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20002cd7-9b23-4038-ad1c-66289a6aa557 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:21.434248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36c99-1379-488d-91f9-d7c54ac75d05 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:21.462351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:21.482798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e261f97-3350-4866-839e-db23810be75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:21.483441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e261f97-3350-4866-839e-db23810be75b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:21.483719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e261f97-3350-4866-839e-db23810be75b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:21.483984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e261f97-3350-4866-839e-db23810be75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:21.510686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:21.511376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:21.511734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:21.512002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:21.512982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:21.513337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:21.514808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:21.539345 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 212/427] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:21 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:21.623365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e261f97-3350-4866-839e-db23810be75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:21.624875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e261f97-3350-4866-839e-db23810be75b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:21.775241 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 216/428] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:42:21.781461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:21.903613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8b70c-e513-4c7d-8b78-82075e2ce7b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:21.943538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:21.945944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:21.947112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:21.947112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:21.947112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] PATH_INFO: `/v3/projects/01418d2a80e444dfa5e16569dd4fed3f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:21.948343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] RBAC: Authorizing `identity:get_project(project_id=01418d2a80e444dfa5e16569dd4fed3f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:21.949797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:21.952199 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 213/429] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:21 2022] GET /identity/v3/projects/01418d2a80e444dfa5e16569dd4fed3f => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:21.957449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e261f97-3350-4866-839e-db23810be75b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:21.983554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:22.068716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e261f97-3350-4866-839e-db23810be75b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:22.114512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc31088-8a15-4ddc-a022-71bada65e2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:22.115201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc31088-8a15-4ddc-a022-71bada65e2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.115504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc31088-8a15-4ddc-a022-71bada65e2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.115773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc31088-8a15-4ddc-a022-71bada65e2f4 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.116994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc31088-8a15-4ddc-a022-71bada65e2f4 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:22.118462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc31088-8a15-4ddc-a022-71bada65e2f4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:22.120633 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 217/430] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:21 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:22.128358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:22.234575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204e8de-a447-432e-9960-caa64d5d0135 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.234944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204e8de-a447-432e-9960-caa64d5d0135 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.235127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204e8de-a447-432e-9960-caa64d5d0135 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.236799 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 218/431] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:22.244675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d97cb-1414-4c32-ba2d-e83dac366a42 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.245042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d97cb-1414-4c32-ba2d-e83dac366a42 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.245235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d97cb-1414-4c32-ba2d-e83dac366a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.262435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a72a9-6896-4046-be3a-907b59416e92 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:22.276889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cb9d97cb-1414-4c32-ba2d-e83dac366a42 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:22.278567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cb9d97cb-1414-4c32-ba2d-e83dac366a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:22.314675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:22.316172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.316626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.317298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.318373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=01418d2a80e444dfa5e16569dd4fed3f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:22.318917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:22.321043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80102bf8-6f1e-4df4-b33a-d46fbe68d020 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:22.329805 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 214/432] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:22 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=01418d2a80e444dfa5e16569dd4fed3f&role.id=9056804b78cc417c8a175569edc8c78a => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:22.368683 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 219/433] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:22.392953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32f34b-769d-47fa-97d4-d5220bd792b0 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.393453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32f34b-769d-47fa-97d4-d5220bd792b0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.393949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32f34b-769d-47fa-97d4-d5220bd792b0 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.395705 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 215/434] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:22.400937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.401222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.401497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.423558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:22.425018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:22.444912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:22.508582 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 220/435] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:22.661769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32f34b-769d-47fa-97d4-d5220bd792b0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:22.726245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:22.767917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32f34b-769d-47fa-97d4-d5220bd792b0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:22.781332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:22.805313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:22.805947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.806215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.806477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.819932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:22.825400 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:22.827211 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 216/436] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:22.832335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:22.901467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17ba55-0f1a-4630-9321-a9fb40f2d6c6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:22.936920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:22.937101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7021421-0fe3-4d9b-8c9f-829bd81c8351 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:22.937483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.937760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.938019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.952988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:22.957886 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:22.958811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:22.959655 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 221/437] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:22.962509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:22.965093 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:22.975497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:22.976101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:22.976380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:22.976665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:22.977332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] RBAC: Adding query filter params (name=member) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:22.977598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:22.978968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:22.983078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:22.988100 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 217/438] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:42:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:22.993493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.080014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a42d9f-909a-4e1f-a862-c4df2203d0dc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.102546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09190223-d24f-4048-ae2a-3db98b457915 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.122938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.123609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.123854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.124124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.124824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:23.125174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.127221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.135958 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 222/439] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:23.140190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19184edf-e065-40ef-b846-9a17d18fb92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.140190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184edf-e065-40ef-b846-9a17d18fb92f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.140410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184edf-e065-40ef-b846-9a17d18fb92f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.142518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19184edf-e065-40ef-b846-9a17d18fb92f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.142631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.201733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.217376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184edf-e065-40ef-b846-9a17d18fb92f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.218850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19184edf-e065-40ef-b846-9a17d18fb92f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.222882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.247771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095a7fb-bbe7-46e9-ab02-9a7de51d71a0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.281552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d24ba9-d590-44c1-973e-0c019ef41154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.282157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d24ba9-d590-44c1-973e-0c019ef41154 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.282430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d24ba9-d590-44c1-973e-0c019ef41154 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.282697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d24ba9-d590-44c1-973e-0c019ef41154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.320149 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 218/440] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:42:23.321447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81892c39-82bf-458d-b502-5eff3867b838 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.321730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81892c39-82bf-458d-b502-5eff3867b838 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.321992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81892c39-82bf-458d-b502-5eff3867b838 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.323232 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 219/441] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:23.328246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81892c39-82bf-458d-b502-5eff3867b838 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.348196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d24ba9-d590-44c1-973e-0c019ef41154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.349658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d24ba9-d590-44c1-973e-0c019ef41154 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.410870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.422830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.428329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81892c39-82bf-458d-b502-5eff3867b838 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.449355 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 223/442] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 11 00:42:23.450383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.450671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.450917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.465426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.465985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.466262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.466524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.467681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.468206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:23.469127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.469396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:23.470744 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 220/443] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:23.471533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.535148 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 224/444] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:23.536008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.569914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c129c87-c278-49e6-9ef4-6b86328dbe64 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.610894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.611557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.611825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.612090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] PATH_INFO: `/v3/projects/d86b1548e5624d668a2cd70a520c24f5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.613195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] RBAC: Authorizing `identity:get_project(project_id=d86b1548e5624d668a2cd70a520c24f5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.614625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.616263 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 221/445] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/projects/d86b1548e5624d668a2cd70a520c24f5 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:23.621697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.643769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.652644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fd248-945e-4521-8f8b-1ff942b35da5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.655574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.689031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.689659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.689933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.690197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.691441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.692874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.694443 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 225/446] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:23.700256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.730703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2b67f-3bad-4a48-8b2e-3438f3c79a59 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.764579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.765206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.765479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.765748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] PATH_INFO: `/v3/groups/365252aeff4640feb74434cf3acf1f66` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.766871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] RBAC: Authorizing `identity:get_group(group_id=365252aeff4640feb74434cf3acf1f66)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.768345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.770099 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 222/447] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/groups/365252aeff4640feb74434cf3acf1f66 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:23.775640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.800838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d4043-61f9-4384-9257-ca56f772f904 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.833889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.834587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.834928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.835222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.835963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=1610cc07f3b24392a709fa07d663cb77, scope.project.id=d86b1548e5624d668a2cd70a520c24f5) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:23.836269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.837721 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.843524 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 226/448] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=1610cc07f3b24392a709fa07d663cb77 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:23.848128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:23.863208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.877356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef077e78-3732-477c-8fca-d5f8265333e2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:23.894033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:23.916908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:23.917568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:23.917848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:23.918113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:23.918876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] RBAC: Adding query filter params (group.id=365252aeff4640feb74434cf3acf1f66, role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=d86b1548e5624d668a2cd70a520c24f5) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:23.919199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:23.920668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:23.927239 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 223/449] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/role_assignments?group.id=365252aeff4640feb74434cf3acf1f66&scope.project.id=d86b1548e5624d668a2cd70a520c24f5&role.id=9056804b78cc417c8a175569edc8c78a => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:23.979382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52501154-1eab-4eaf-a162-a0a3c17dbf70 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:24.019713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:24.020259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:24.020528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:24.020790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:24.033183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:24.038091 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:24.039253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:24.040253 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 227/450] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:24.045660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:24.157428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31102ec5-57ec-43e7-b425-9315678253f4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:24.192940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:24.193550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:24.193821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:24.194093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:24.194754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:24.195049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:24.196647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:24.206362 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 224/451] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:24.212405 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:24.310510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:24.317369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49955c-0ee7-47f9-ad13-31e96dfc4503 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:24.351381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:24.351987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:24.352262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:24.352535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:24.419732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:24.421204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:24.515051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:24.527273 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 228/452] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:24.536290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:24.556806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:24.647915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a0904f-c3b2-4ad5-b0eb-a5a47365ab67 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:24.682950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:24.683696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:24.683972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:24.684232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] PATH_INFO: `/v3/projects/857538f273814e30bf74c0248d309754` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:24.685753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] RBAC: Authorizing `identity:get_project(project_id=857538f273814e30bf74c0248d309754)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:24.687201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:24.688805 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 225/453] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:42:24 2022] GET /identity/v3/projects/857538f273814e30bf74c0248d309754 => generated 344 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:24.694850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:24.807779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d06b4-741f-4a62-bfd6-5ee89cec8f94 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:24.843956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7faae87-4d45-4958-adfa-63f81cab9936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:24.844555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faae87-4d45-4958-adfa-63f81cab9936 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:24.844824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faae87-4d45-4958-adfa-63f81cab9936 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:24.845088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7faae87-4d45-4958-adfa-63f81cab9936 None admin] PATH_INFO: `/v3/groups/8ef3c4b63f8746b697af4df200888711` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:24.846214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faae87-4d45-4958-adfa-63f81cab9936 None admin] RBAC: Authorizing `identity:get_group(group_id=8ef3c4b63f8746b697af4df200888711)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:24.847724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7faae87-4d45-4958-adfa-63f81cab9936 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:24.849257 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 229/454] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Sep 11 00:42:24 2022] GET /identity/v3/groups/8ef3c4b63f8746b697af4df200888711 => generated 229 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:24.858911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:24.951493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:24.970148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a9aa6-3c26-4189-8542-6d3be3ad7481 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:25.003553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:25.004143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:25.004413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:25.004680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:25.005401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] RBAC: Adding query filter params (group.id=8ef3c4b63f8746b697af4df200888711, role.id=77132358e0b64de4a63750b81e359340, scope.project.id=857538f273814e30bf74c0248d309754) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:25.005710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:25.007132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18a39a-d833-43ff-b5d7-f132f2c886bf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:25.013039 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 226/455] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Sep 11 00:42:24 2022] GET /identity/v3/role_assignments?group.id=8ef3c4b63f8746b697af4df200888711&scope.project.id=857538f273814e30bf74c0248d309754&role.id=77132358e0b64de4a63750b81e359340 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:27.571148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4332a-e107-4680-9f66-e726c64954af None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:27.572303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4332a-e107-4680-9f66-e726c64954af None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:27.572773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4332a-e107-4680-9f66-e726c64954af None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:27.574584 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 230/456] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:27.580460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b20c3-2b56-41d5-a5e9-96a290d356c9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:27.581360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b20c3-2b56-41d5-a5e9-96a290d356c9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:27.581817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b20c3-2b56-41d5-a5e9-96a290d356c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:27.611457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ca6b20c3-2b56-41d5-a5e9-96a290d356c9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:27.613114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ca6b20c3-2b56-41d5-a5e9-96a290d356c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:27.709858 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 227/457] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:27.902699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcc011-6fee-4726-84fd-2d3cf3b08d70 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:27.903045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcc011-6fee-4726-84fd-2d3cf3b08d70 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:27.903276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcc011-6fee-4726-84fd-2d3cf3b08d70 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:27.904917 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 231/458] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:27.911273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e51eca-607f-4fe4-bb3d-ba4a72f42c9d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:27.911555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e51eca-607f-4fe4-bb3d-ba4a72f42c9d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:27.911818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e51eca-607f-4fe4-bb3d-ba4a72f42c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:27.940152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e4e51eca-607f-4fe4-bb3d-ba4a72f42c9d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:27.941446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e4e51eca-607f-4fe4-bb3d-ba4a72f42c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:28.033877 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 228/459] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:28.075893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcc011-6fee-4726-84fd-2d3cf3b08d70 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.150907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f0c55-3bf1-4f6e-bf24-67108afe541f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.151262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f0c55-3bf1-4f6e-bf24-67108afe541f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.151471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f0c55-3bf1-4f6e-bf24-67108afe541f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.153019 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 229/460] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:28.158005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47d1b1-7b3d-4639-bd6a-696614c83a22 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.158250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47d1b1-7b3d-4639-bd6a-696614c83a22 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.158871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47d1b1-7b3d-4639-bd6a-696614c83a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.174057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:28.192034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fa47d1b1-7b3d-4639-bd6a-696614c83a22 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:28.193354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fa47d1b1-7b3d-4639-bd6a-696614c83a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:28.195508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:28.228919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcc011-6fee-4726-84fd-2d3cf3b08d70 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:28.265465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:28.266373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.266782 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.267196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.278807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:28.280778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8d719f0a-802c-4a66-a85f-65d6f24b4211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:28.296798 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:28.297943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:28.299163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:28.299163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:28.299163 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:28.299396 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 230/461] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:28.300641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398711b4-3f3e-4a83-9491-f6c367ba303f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.301014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398711b4-3f3e-4a83-9491-f6c367ba303f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.306007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398711b4-3f3e-4a83-9491-f6c367ba303f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.306007 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 231/463] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:28.306217 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 232/463] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:28.308839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398711b4-3f3e-4a83-9491-f6c367ba303f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.313023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.313297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.313561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.331815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:28.332996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:28.410607 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 233/464] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:28.412460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.446186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398711b4-3f3e-4a83-9491-f6c367ba303f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:28.505671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:28.519017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:28.519742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.520018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.520287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.520988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:28.521292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:28.522675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:28.528781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:28.547258 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 232/465] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:28.552257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.613916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92cd6c-8196-4d9c-974a-d9d8556b7841 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:28.647400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:28.648430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.648907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.649259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.659576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:28.661180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:28.676118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:28.677271 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:28.678349 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:28.678349 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:28.678349 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:28.680636 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 234/466] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:28.681671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.696520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4edba0f-776e-473b-885e-4286958cb755 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:28.742481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:28.743506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.744207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.744663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.746219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:28.748043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:28.771858 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 233/467] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:28 2022] POST /identity/v3/users => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:28.772943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.799931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:28.805973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104222-85f6-48b9-83a5-a978e44d6812 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:28.855361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:28.856373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:28.857033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.857330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.857561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.871335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:28.877831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:28.899826 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:28.900815 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:28.901675 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:28.901675 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:28.901675 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:28.906140 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 235/468] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:28.907106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:28.945523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c4075-e96f-4a91-888a-77edd4b9bf38 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:28.990085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:28.990915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:28.991259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:28.991476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:28.992126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:28.992422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:28.993795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.008456 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 234/469] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:29.009239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:29.018892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000729aa-2c92-4eb1-aa0e-ba9d42a7a529 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:29.069999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:29.070635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.070909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.071204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.083358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:29.084811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:29.099709 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:29.100670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:29.107275 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:29.107275 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:29.107275 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:29.107275 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 236/470] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:29.107275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:29.109282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:29.147812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2d64e-4c51-4df7-8f35-3e85a7b13dfe None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:29.163292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:29.188196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:29.189224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.189781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.190311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.191411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:29.191933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:29.193559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.207003 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 235/471] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:29.208215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:29.216698 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa9662-4496-4e57-9f6d-17ecbec412ca None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:29.253221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:29.253830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.254124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.254364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.255012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:29.256587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.288286 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 237/472] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:29 2022] POST /identity/v3/users => generated 248 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:29.288286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:29.337930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a541492-6097-4b54-b3ad-e84a8db24231 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:29.370005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:29.381143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:29.381771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.382042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.382309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.382999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:29.383388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:29.384968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1f731-cecc-421d-ad55-9bcd97c68589 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.399495 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 236/473] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:29.400824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42991db-3fe9-4c42-9102-53d7039a5953 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.401014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42991db-3fe9-4c42-9102-53d7039a5953 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.401301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42991db-3fe9-4c42-9102-53d7039a5953 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.402862 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 237/474] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:29.406057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42991db-3fe9-4c42-9102-53d7039a5953 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:29.423403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf227d-6e18-46c3-a57a-71cea638c526 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:29.459652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:29.470080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2b48c-e4c7-4f00-a27e-677b0cf3ffd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:29.470706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b48c-e4c7-4f00-a27e-677b0cf3ffd7 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.470978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b48c-e4c7-4f00-a27e-677b0cf3ffd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.471271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2b48c-e4c7-4f00-a27e-677b0cf3ffd7 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.472212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2b48c-e4c7-4f00-a27e-677b0cf3ffd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:29.474924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2b48c-e4c7-4f00-a27e-677b0cf3ffd7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.504671 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 238/475] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:29 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:29.506243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.506527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.506802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.528795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:29.530446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:29.537129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42991db-3fe9-4c42-9102-53d7039a5953 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:29.581533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:29.582268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:29.582620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:29.582959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:29.583821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:29.585470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:29.620819 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 238/476] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:29 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:29.634646 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 239/477] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:29.911540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:30.032535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:30.048129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2279e-b5b7-44a2-8336-ac9ddb202a19 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:30.092559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:30.093556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:30.094024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:30.094500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:30.095402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:30.096997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:30.108150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 00:42:30.108150 dv-cinder-master-3-smb devstack@keystone.service[81488]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 00:42:30.108150 dv-cinder-master-3-smb devstack@keystone.service[81488]: [parameters: {'id': 'c4a8a84fb5044db38a7b819b1424d38c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 00:42:30.108150 dv-cinder-master-3-smb devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Sep 11 00:42:30.131395 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return ref.to_dict() Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 00:42:30.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 00:42:30.133326 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 00:42:30.134277 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 00:42:30.135994 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 00:42:30.137956 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': 'c4a8a84fb5044db38a7b819b1424d38c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:30.139895 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 11 00:42:30.141817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:30.145367 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 239/478] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Sep 11 00:42:29 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:30.152381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:30.243391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:30.321981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c304ecc-3c77-447a-9bff-e0fef8da1d6e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:30.364768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b93bde-31c8-4854-ad75-598d946cac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:30.365408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b93bde-31c8-4854-ad75-598d946cac3a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:30.365679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b93bde-31c8-4854-ad75-598d946cac3a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:30.365946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b93bde-31c8-4854-ad75-598d946cac3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:30.380034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b93bde-31c8-4854-ad75-598d946cac3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-27b93bde-31c8-4854-ad75-598d946cac3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:30.384828 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:30.386813 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 11 00:42:30.388741 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:30.394779 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 240/479] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:42:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:30.394979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:30.522324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f723be-aaa5-45f4-9e21-629fa8d6935b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:30.558144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:30.559145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:30.559998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:30.560455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:30.561335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:30.561838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:30.563431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba89b9b-304d-4759-a68d-b3c5ed058c39 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:30.572718 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 240/480] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:31.230965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e65b0-4240-4081-954d-fae369d16cf7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:31.231981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e65b0-4240-4081-954d-fae369d16cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:31.232472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e65b0-4240-4081-954d-fae369d16cf7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:31.234288 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 241/481] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:31.239711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:31.240544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:31.241023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:31.268157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:31.269777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:31.279889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:31.354137 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 241/482] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:31.822545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e65b0-4240-4081-954d-fae369d16cf7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:31.963877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e65b0-4240-4081-954d-fae369d16cf7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:31.966337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:32.016676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd60baf-fa3a-404a-b57a-636490157199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.017344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd60baf-fa3a-404a-b57a-636490157199 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.017616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd60baf-fa3a-404a-b57a-636490157199 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.017885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd60baf-fa3a-404a-b57a-636490157199 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.039163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd60baf-fa3a-404a-b57a-636490157199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0cd60baf-fa3a-404a-b57a-636490157199 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:32.043982 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:32.044863 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:32.046118 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 242/483] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:32.056297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.106662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79898-8bbf-4083-920c-86f0a687ea79 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.107022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79898-8bbf-4083-920c-86f0a687ea79 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.107228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a79898-8bbf-4083-920c-86f0a687ea79 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.108885 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 243/484] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:32.113850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.114124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.114389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.132298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:32.133494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:32.231201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b1b7f-422d-4c8e-9083-73a2ad4067d8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:32.278062 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 244/485] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:32.300636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.301277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.301525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.301788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.302521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:32.302819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:32.304216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98895ec3-9fef-44de-95a2-3cb4f28ad304 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:32.317785 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 242/486] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:32.323307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.331085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9382b-71f2-4060-ba14-8d5a16b24b9b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.331404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9382b-71f2-4060-ba14-8d5a16b24b9b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.331730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab9382b-71f2-4060-ba14-8d5a16b24b9b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.333385 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 243/487] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:32.346805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20c3a-39ab-4a52-9cb8-c239732fa131 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.346805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20c3a-39ab-4a52-9cb8-c239732fa131 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.346805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a20c3a-39ab-4a52-9cb8-c239732fa131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.362331 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:32.372700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c7a20c3a-39ab-4a52-9cb8-c239732fa131 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:32.373933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c7a20c3a-39ab-4a52-9cb8-c239732fa131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:32.434159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:32.441744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a23ddc-cdee-4db8-8e1f-cad0fd2350c7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:32.468043 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 244/488] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:32.469890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abadeb3-873d-44b5-84e7-89ab77dea4f8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.470427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abadeb3-873d-44b5-84e7-89ab77dea4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.470709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abadeb3-873d-44b5-84e7-89ab77dea4f8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.475322 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 245/489] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:32.477904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.478180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.478446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.487413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.488401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.488883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.489361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.498151 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:32.499555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:32.501624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:32.503491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:32.519386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:32.520353 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:32.521238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:32.521238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:32.521238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:32.523326 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 245/490] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:32.529332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.607204 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 246/491] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:32.607204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.672582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a9820-1dcd-428b-bda3-c8c5049ac18c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:32.704255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:32.748505 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.749116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.749390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.749657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.750304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:32.750602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:32.753673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c97d2f-51d0-4aac-8285-03aef553f86d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:32.755518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:32.778524 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 246/492] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:32.784661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.795071 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:32.808070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.809676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.809955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.810221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.828190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:32.835107 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:32.836029 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:32.836847 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 247/493] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:32.837976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.901296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5acc1e-cf7f-4e5f-a0c5-11a3c3e06fe0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:32.936175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.936718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.936985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.937243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:32.944794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:32.946211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:32.951113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ccfa2-573f-4a97-bcf8-137ef701803e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:32.958596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:32.959760 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:32.960720 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 247/494] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:32.961307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:32.983836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:32.998184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:32.998815 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:32.999084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:32.999372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.009860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:33.015131 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:33.016095 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:33.016918 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:33.025242 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 248/495] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:33.026648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.032979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.077654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfaa43-e6bb-4796-8242-0cc6ca60cca8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.117712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.118601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.118915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.119145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.119861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:33.120160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.122165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.130396 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 248/496] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:33.131195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.149048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fdfbf-0168-4140-a042-aba04518b600 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.185499 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.186131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.186485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.186673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.197773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:33.203142 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:33.204063 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:33.204880 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 249/497] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:33.205549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.248475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c963b5d-4233-47bb-944c-723c9fdac68a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.252946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.278938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.287718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.288362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.288634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.288901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.289711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:33.289918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.291388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.313892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18d1c-455f-4f55-aeee-e27e3208a543 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.336878 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 249/498] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:33.336878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.359902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.360521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.360793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.361052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.361722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:33.362018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.363535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.373495 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 250/499] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:33.379509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.447583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf0975-c1d6-4b88-98a1-22c6ac5bcc90 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.481217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3d7ce-b4a3-40c4-989f-18972a02ae4f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.483444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.484069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.484340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.484600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.490062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.494786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.496253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:33.510383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:33.511367 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:33.512404 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:33.512404 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:33.512404 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:33.512404 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 250/500] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:33.512404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.518152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.518785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.519053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.519352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.520017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:33.520295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.521646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.530252 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 251/501] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:33.531119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.554603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.613751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf9993-9179-4b13-90b1-a3b82412ff1c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.630298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76616-5dc3-438d-b4bf-8cc83e3b2240 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.647709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.648317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.648594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.648859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.662573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.663353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.663656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.663905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.674344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.675916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f86789cc-7098-439f-9db9-d98641579b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:33.690240 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:33.691259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:33.692116 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:33.692116 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:33.692116 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:33.692116 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 252/502] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:33.692686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd6f24-f926-4cee-a0be-9d69f234ff38 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.692786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd6f24-f926-4cee-a0be-9d69f234ff38 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.692981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd6f24-f926-4cee-a0be-9d69f234ff38 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.694340 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 253/503] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:33.694862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd6f24-f926-4cee-a0be-9d69f234ff38 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.718516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.719440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.720070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.758825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.796354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd6f24-f926-4cee-a0be-9d69f234ff38 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.817947 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 251/504] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:33.819606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.831401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.831998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.832269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.832537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.833172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:33.833464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.834839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.846520 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 254/505] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:33.847096 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:33.919455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05956fd-4ec0-4121-ad20-df28ca35ac46 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.923004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.941215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d23c4c-6bc1-4f77-9aa5-519c20206c20 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:33.952611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.953230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.953497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.953910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.961738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:33.966050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.967533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.973805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:33.974344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.974615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.974880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.975555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:33.975855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:33.977261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a0724836-64b1-42f0-b470-317ab35a6f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:33.984580 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:33.985470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:33.986596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:33.986596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:33.986596 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:33.986596 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 252/506] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:33.987017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:33.987017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:33.987349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:33.997104 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 255/507] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:33.997104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.005098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:34.006284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:34.071796 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 253/508] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:34.072609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.114241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8fe773-7b53-4ef2-b40c-84ac27616499 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.156620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.157503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.157784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.158424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.169820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.171386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.173022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437d71e-4399-49bc-8d59-f8adf4ebe7f0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.173269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:34.187269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:34.188380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:34.189436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:34.189436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:34.189436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:34.195976 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 256/509] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:34.195976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.200911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.215698 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.216495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.216734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.217063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.224455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.225884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:34.237427 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:34.238335 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 254/510] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:34.239255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.299757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a744a3-8df7-41a5-8206-0bb9982c23a5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.334513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.335137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.335406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.335670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.336322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:34.336615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.337982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.340799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da995bc6-7628-42fe-8df2-3812e6075942 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.349868 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 257/511] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:34.350664 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.378680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.379332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.379601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.379863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.389857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.395468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.407248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:34.423227 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:34.424183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:34.425192 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 255/512] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:34.430612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.470633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6b258-f754-42f4-a3e4-ebb5d9bca945 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.473963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.505119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.505652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.505926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.506200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.506837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:34.507137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.508540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.519632 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 258/513] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:34.520413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.534078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e190d7c-d3af-4fd5-910e-b7b750fc184c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.570887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.571526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.571786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.572052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.573203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:34.573507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.575150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.592836 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 256/514] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:34.593635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.619973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d35bb8-0f1e-4857-8d50-49ae28a5f29f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.633359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.654283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.654918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.655229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.655499 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.665885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.667359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:34.681085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:34.682087 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:34.688252 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:34.688252 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:34.688252 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:34.688252 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 259/515] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:34.688252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.693130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.697495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067e22-e225-4ada-98bd-6d7bfcc74802 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.738624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.739296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.739567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.739834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.747209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.748670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:34.761209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:34.762341 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 00:42:34.763268 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:34.767355 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 257/516] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:34.768277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.794170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd835b6-2284-40fd-84ce-ac0c40c81eb6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.828257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.829110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.829376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.829640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.830368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:34.830660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.832115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.867746 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 260/517] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:34.868024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f01bed-fb4c-483a-af27-d6893d3c890f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:34.868423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.893798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.902047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:34.902678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:34.902949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:34.903228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:34.910750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:34.912253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:34.923896 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:34.924811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:34.925695 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 258/518] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:34.926157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:34.945267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:34.971939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17277b-8636-46d4-b6bd-07ff614f9fbd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.005665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.006262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.006534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.006797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.024563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40c788-80fd-4f29-bcfd-14621b0e99ce None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.059207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.060217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.060497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.060764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.061400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:35.061697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.063059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.075877 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 259/519] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:35.076669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.078478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.080083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.105502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.160494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.182572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360cea4-41e8-4280-96c1-030b69a09853 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.183973 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 261/520] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:35.185160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.218661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.219304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.219581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.219847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.220576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:35.220881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.222334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.238838 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 260/521] 10.222.0.23 () {56 vars in 1139 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:35.239673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.285548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a5930-43ea-46f0-a384-cf08deb3c03a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.308236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.319690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.320294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.320564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.320832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.335304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55990b-3ff2-43e4-b85a-b6225903319b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.369918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.370621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.370972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.371265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.371946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:35.372245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.373692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.377729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.388610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.390247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.392665 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 261/522] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:35.393603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.487209 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 262/523] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:35.487209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.494011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1fa16-9ffe-44f4-9028-1d94bba53c92 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.510576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.528650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.529256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.529552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.529805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.530446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.531920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.553564 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 262/524] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:35 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:35.554305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.611720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.615689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441950ed-c8b8-437b-a5fe-cca6cdfaba10 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.655354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.656002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.656274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.656538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.667834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.669308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.671434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f48b18-c377-40de-a04b-5e89ec6e0b77 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:35.685621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:35.686691 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:35.688054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:35.688054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:35.688054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:35.688054 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 263/525] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:35.688054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.716146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.717150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.717534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.717803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.774572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.801029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aebadf-fbbe-4d8a-9270-658567a27947 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:35.801256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.802529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.836580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:35.837285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:35.837553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:35.837963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:35.838679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=c150143af9144547ab3dc5a5d318d45f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:35.838977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:35.840424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:35.846167 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 264/526] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/role_assignments?user.id=c150143af9144547ab3dc5a5d318d45f&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:35.847072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.866230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:35.924792 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 263/527] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:35.926206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:35.963660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6940f-4432-4497-9aed-195f61b3c3f5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.006800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.007655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.008052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.008584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.023479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.025868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.028429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:36.043585 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:36.044833 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:36.045869 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:36.045869 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:36.045869 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:36.045869 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 265/528] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:36.046159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.055159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9cec9a-3408-4213-8441-377fb3fd5f93 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.096336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.097341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.097767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.098043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.098748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:36.099099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.100532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.113557 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 264/529] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:36.114462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.131658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.173817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7639e-cf65-41bd-bc80-57b395994150 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.218069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.218861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.219303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.219672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.233882 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.235544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.250456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c6847-55c0-48ff-81ba-29b7ea6a3c05 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:36.254568 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:36.255610 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:36.256632 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:36.256632 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:36.256632 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:36.256632 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 266/530] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:36.257604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.289295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.290565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.291334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.291773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.292582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:36.293044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.294743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.308863 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 265/531] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:36.310462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.315756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.382875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c820b-6314-4164-a6df-71ed80784972 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.397504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.422773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.424190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.424709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.425206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.432860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.433797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338585c2-d14e-45a4-bce7-406abc3f8ac5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.435063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:36.447882 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:36.448884 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:36.449737 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:36.452195 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 267/532] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:36.453126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.477144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.477865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.478139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.478398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.479039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:36.479383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.480779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.498130 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 266/533] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:36.499765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.579005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a304a-17c9-43da-bd35-a8edf52b951b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.582347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.616724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.617350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.617625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.617883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.625259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.626862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.638373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1272e20d-2f1c-4c8a-87fb-76003a88b805 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:36.638520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:36.639354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:36.640215 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 268/534] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:36.641300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.682107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.689363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.689985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.690486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.690736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.691478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:36.691782 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.693323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.707416 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 267/535] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:36.708330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.746618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8050ac0-536e-44a4-a87e-47c8b1bef0ee None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.782410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.783040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.783581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.783855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.792035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.793960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:36.808958 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:36.810473 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:36.811558 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 269/536] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:36.812111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.820080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf574b-46dd-4d7b-993d-14b4ac906282 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.846754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.861636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.862153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.862435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.862702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.863426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:36.863731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.865218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:36.877458 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 268/537] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:36.878142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:36.933214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:36.948651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9399f7-9ea4-47cc-8137-3fbe3bc81586 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.990804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7cc8-143d-4380-a10a-b8a6925a037e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:36.992174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:36.992785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:36.993059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:36.993410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:36.994051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=8388bedc588f4f5f94f1e732d46e83b7) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:36.994355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:36.995993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:37.001893 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 270/538] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/role_assignments?user.id=8388bedc588f4f5f94f1e732d46e83b7&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:37.002808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:37.038845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:37.040086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:37.040562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:37.041878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:37.042751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:37.043265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:37.044877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:37.063780 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 269/539] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:37.074763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:37.143859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:37.172155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae561e56-fe67-46b2-86eb-ade633dc7b64 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:37.227500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:37.227823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:37.228075 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:37.228339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:37.229078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=7d71b946e7d64a50888cfcc337d688a5) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:37.229383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:37.230834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c431c0-bdd4-43c7-8e58-9b2b7ae24144 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:37.243652 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 271/540] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:37 2022] GET /identity/v3/role_assignments?user.id=7d71b946e7d64a50888cfcc337d688a5&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:37.269217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad29d62-4a71-4e96-af19-fef79f4cccb1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:37.283915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:37.312971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:37.313603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:37.313875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:37.314140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:37.314890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=341595ae0d9b4431b4417edda29f03ff) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:37.315272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:37.316787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a42b6ba-36d0-4df7-a468-7cd9ab2d5739 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:37.328283 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 270/541] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:37 2022] GET /identity/v3/role_assignments?user.id=341595ae0d9b4431b4417edda29f03ff&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:37.892809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2943bed-1dc9-4eeb-87aa-30bc047a2f2b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:37.893402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2943bed-1dc9-4eeb-87aa-30bc047a2f2b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:37.893482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2943bed-1dc9-4eeb-87aa-30bc047a2f2b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:37.896178 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 272/542] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:37.901923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128828a-54f1-4305-af95-d5359b420638 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:37.902381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128828a-54f1-4305-af95-d5359b420638 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:37.902479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128828a-54f1-4305-af95-d5359b420638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:37.931578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0128828a-54f1-4305-af95-d5359b420638 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:37.932787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0128828a-54f1-4305-af95-d5359b420638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:38.024602 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 271/543] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:38.245597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07c314-e992-48f5-9558-0216e747f1c0 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:38.245931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07c314-e992-48f5-9558-0216e747f1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:38.246134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c07c314-e992-48f5-9558-0216e747f1c0 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:38.247806 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 273/544] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:38.253138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:38.253419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:38.253687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:38.280163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:38.281427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:38.361318 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 272/545] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:38.377978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07c314-e992-48f5-9558-0216e747f1c0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:38.504962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c07c314-e992-48f5-9558-0216e747f1c0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:38.549861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:38.550585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:38.550905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:38.551234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:38.569330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:38.574661 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:38.575666 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:38.576488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:38.578781 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 274/546] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:38.584846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:38.663118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:38.690580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:38.720028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:38.752702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104dd628-0d50-4a75-ad7e-b56fe39466a6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:38.790478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:38.791010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:38.791312 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:38.791588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:38.792241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:38.792539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:38.793910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:38.812887 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 273/547] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:38.819014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:38.864005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3037ade-b9bd-4282-9e40-d5c2f10d7c13 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:38.900582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:38.902027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:38.902501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:38.902963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:38.914939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:38.921009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:38.921928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:38.922746 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:38.925831 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 275/548] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:38.936344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:38.984025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:38.990076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b61628-3b48-4a85-b75d-5bd381821421 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.041113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.041672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.041945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.042442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.058138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.060167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:39.075811 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:39.077302 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:39.078339 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:39.078339 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:39.078339 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:39.078339 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 274/549] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:39.086747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:39.087951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.118487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bce8fb-21b4-43a5-9ce1-58eee17eff69 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.161757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.162376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.162652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.162926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.163636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:39.163939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.165441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.173926 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 276/550] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:39.182597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.253684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a146b34-43ac-4dfd-b0ae-4d44e4e56bc6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.291024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.291709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.291920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.292183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.293216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:39.293524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.295207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.310196 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 275/551] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:39.323268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.331620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:39.359919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd472f9c-47fc-4171-99e2-a6e3b3276042 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.433368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.434008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.434636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.434903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.447489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:39.462493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.463985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:39.488089 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:39.489204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:39.490098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:39.490098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:39.490098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:39.490265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449183e-ae2d-4b6a-8441-771216294a04 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.492301 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 277/552] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:39.498311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.551828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.552941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.553414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.553892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.564961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.566461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-760875a6-9b67-4970-ba46-b6d41300cefa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:39.586995 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:39.588042 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:39.589634 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 276/553] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:39.590218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef94fb-a313-4d22-83e9-7c0ef67154d6 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.590218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef94fb-a313-4d22-83e9-7c0ef67154d6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.590573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef94fb-a313-4d22-83e9-7c0ef67154d6 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.591890 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 277/554] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:39.593607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef94fb-a313-4d22-83e9-7c0ef67154d6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.652307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474086c-ee84-4327-a025-2db576dcbe9f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.670263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:39.700309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.701400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.701842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.702610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.703701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:39.704013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.705398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f295cf8-9cf6-4ffc-a4a5-ea184320989c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.707828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef94fb-a313-4d22-83e9-7c0ef67154d6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.722671 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 278/555] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:39.722671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.722671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.722671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.739018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:39.740265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:39.756015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.757140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.757552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.757970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.758999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:39.759722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.761275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.783460 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 278/556] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:39.785073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.787072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:39.814586 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 279/557] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:39.815643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:39.908151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6c68a-7c19-4001-b75a-89fdd6b4815b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:39.944855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:39.956046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:39.956956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:39.957438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:39.957893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:39.966236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:39.968109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:39.981289 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:39.982738 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:39.983700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:39.987609 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 279/558] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:39.994780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.048088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46318a-3835-4069-848c-3da13b36b362 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.114802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ec42f-32d9-4242-a4e7-b1c6aba02ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.115835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ec42f-32d9-4242-a4e7-b1c6aba02ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.116371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ec42f-32d9-4242-a4e7-b1c6aba02ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.116644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ec42f-32d9-4242-a4e7-b1c6aba02ec3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.118118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8647fa5-7c58-45a1-b137-899125c09b9c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.153008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.153618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.153889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.154150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.154869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:40.155164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.156600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.166605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.179394 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 280/559] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:40.180041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.195221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ec42f-32d9-4242-a4e7-b1c6aba02ec3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.196928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ec42f-32d9-4242-a4e7-b1c6aba02ec3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.228607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.280512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139036c-029e-4bc5-b85d-dfa02bd35dde None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.296749 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 280/560] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:40.298016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9488-ffd4-48f6-93e1-00dc8d15ac1b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.298296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9488-ffd4-48f6-93e1-00dc8d15ac1b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.298563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f9488-ffd4-48f6-93e1-00dc8d15ac1b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.299944 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 281/561] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:40.300627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f9488-ffd4-48f6-93e1-00dc8d15ac1b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.319398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2614133-52ce-451c-bdf3-70696dce78df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.320033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614133-52ce-451c-bdf3-70696dce78df None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.320306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614133-52ce-451c-bdf3-70696dce78df None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.320575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2614133-52ce-451c-bdf3-70696dce78df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.330546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2614133-52ce-451c-bdf3-70696dce78df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f2614133-52ce-451c-bdf3-70696dce78df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:40.335375 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:40.336470 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:40.337391 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 281/562] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:40.338176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4b838-fb9c-4964-8ace-b7f59e12669d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.338176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4b838-fb9c-4964-8ace-b7f59e12669d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.338455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4b838-fb9c-4964-8ace-b7f59e12669d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.339762 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 282/563] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:40.340309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4b838-fb9c-4964-8ace-b7f59e12669d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.386887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.411023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f9488-ffd4-48f6-93e1-00dc8d15ac1b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.440188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4b838-fb9c-4964-8ace-b7f59e12669d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.446008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.446637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.446911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.447170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.447424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.457267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.458704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-69e689b5-75e6-441a-93e0-66de92aee5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:40.472633 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:40.473658 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:40.474577 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:40.474577 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:40.474577 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:40.474577 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 282/564] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:40.474916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debbe8-928a-4cf6-b065-eb85a40a55f9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.475261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debbe8-928a-4cf6-b065-eb85a40a55f9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.475428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10debbe8-928a-4cf6-b065-eb85a40a55f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.479949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.480576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.480847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.481111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.491606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.493032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.494796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-10debbe8-928a-4cf6-b065-eb85a40a55f9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:40.496019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-10debbe8-928a-4cf6-b065-eb85a40a55f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:40.507238 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:40.508359 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:40.509800 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:40.509800 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:40.509800 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:40.509800 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 283/565] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:40.509800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.565469 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 283/566] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:40.566819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.567098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.567391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.584947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:40.586126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:40.626150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2dfb5-a775-4172-8a32-643885d8e6b7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.641092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.656348 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 284/567] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:40.657054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.667880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.669225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.670109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.670603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.671765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:40.672533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.674891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.687593 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 284/568] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:40.688702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.716563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.764311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38608f-83ed-4a1c-a1ed-ad18eee92e3d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.798350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.798952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.799293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.799614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.800246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:40.800616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.801906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.818143 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 285/569] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:40.818143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.830199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d710f2f7-95cd-48f8-a3ec-0dcd5f0a0a1f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.867679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.868292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.868563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.868820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.869464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:40.869760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.870493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.871207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.887644 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 285/570] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:40.887644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:40.930434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ec3b-adf2-4939-8fa1-8f75bee9dc74 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.963249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:40.963732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:40.964451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:40.964694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:40.964892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:40.975282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:40.976729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:40.984477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f92dd-0f68-4bc3-9369-ea0670839bf1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:40.990811 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:40.991925 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:40.992858 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:40.992858 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:40.992858 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:40.992858 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 286/571] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:40.992858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.022796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.023470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.023743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.024012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.031048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.032518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:41.043735 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:41.044653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:41.045539 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 286/572] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:41.046238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.093944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.097264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bb171-d846-41a3-bc12-71bc41272ed7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.130112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.130733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.131004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.131298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.141796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:41.144553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbeb6-11a6-463a-ae5e-235be8cf1c1b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:41.147219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:41.148227 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 287/573] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:41.149108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.181614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.182183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.182454 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.182718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.190036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.191512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.193902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:41.203425 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:41.204451 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 287/574] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:41.205394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.257911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a2050-976d-40a9-9e15-7b09e3a49085 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.290892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.291544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.291805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.292068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.302101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:41.303223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be996756-8721-4b7a-9ddb-9e88ab2dccb6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:41.307304 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:41.308239 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:41.309120 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 288/575] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:41.310196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.333973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.339342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.339992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.340268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.340537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.341185 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:41.341521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.342899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.354672 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 288/576] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:41.355487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.410660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2b6b7-d302-4c04-a2c1-5e432c0b38b3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.416087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.443771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.444292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.444560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.444823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.445498 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:41.445789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.447220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.456866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ece3d-f408-4291-8273-80582923df4e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.465562 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 289/577] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:41.466364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.491259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.491876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.492144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.492407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.493057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:41.493341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.494709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.502346 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 289/578] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:41.503369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.547922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.567483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d88f6-2046-4091-92c6-00244f06c6cc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.600361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.600974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.601275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.601547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.602232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:41.602535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.603520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4044-6532-4e64-a5f7-fd486c21acb5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.604000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.615539 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 290/579] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:41.616334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.630772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.642639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.643325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.643599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.643980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.644676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:41.644975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.646336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.654090 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 290/580] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:41.654840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.714138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be658b3e-c5b5-42b5-b5eb-e04f36e0dad7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.746065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.746689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.746960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.747256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.750988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d03acb-64d1-4a6a-b36e-d75254534399 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.754671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.756132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.758651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:41.769370 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:41.770547 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 291/581] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:41.771562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.789545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.790206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.790481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.790746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/c150143af9144547ab3dc5a5d318d45f/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.804773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=c150143af9144547ab3dc5a5d318d45f, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.806207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.819107 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 291/582] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:42:41 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/c150143af9144547ab3dc5a5d318d45f/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:41.819928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.856019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:41.892313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f305128-0aaf-4c05-83c7-d646328195cb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.928601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2bca7b-856a-41e7-b2b9-0766dcb1ec79 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:41.928987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.929635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.929911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.930184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.942152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.943677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:41.959288 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:41.960503 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:41.961429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:41.961429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:41.961429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:41.961429 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 292/583] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:41.961429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:41.976024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:41.976901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:41.977334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:41.977610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:41.978371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=512080b44c7542439333bd8b5f941496) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:41.978678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:41.980158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:41.986485 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 292/584] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/role_assignments?user.id=512080b44c7542439333bd8b5f941496&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:41.987436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.045608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.102762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae2d32-9fc2-40cd-82cc-e68e621ac0e1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.125355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.130638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85e7dc-c959-4744-8451-b5315ab5433a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.145724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.146693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.147252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.147777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.164618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.166426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.172570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.173190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.173465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.173738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.174537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:42.174853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.176701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:42.185285 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:42.186261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:42.187166 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:42.187166 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:42.187166 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:42.189520 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 293/585] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:42.190681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.206035 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 293/586] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Sep 11 00:42:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:42.206035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.302704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd8e2d-eeff-4422-8acb-2960b69a9734 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.306344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.327205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928664-ca8e-4e9c-bd02-9e1841cda0cb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.342630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.343535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.343811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.344454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.345108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:42.345414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.347464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.362465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.363082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.363415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.363518 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 294/587] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:42.363672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.364296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.364401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:42.364595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.365977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.394202 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 294/588] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:42.395114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.415377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.496066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd31a22-4f3e-47a6-8cde-a83a0281c26c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.513323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810cc00-a95a-46c0-a05d-74458485e28d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.530780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.531614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.532039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.532400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.543094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.544736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.553811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.555108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.555579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.556359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:42.562381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:42.563707 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:42.564825 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:42.564825 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:42.564825 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:42.565547 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 295/589] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:42.570309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.571569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.573149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:42.587639 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:42.588712 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:42:42.589666 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:42.593813 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 295/590] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:42.594808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.603536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.684925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.711864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843812b-b0b7-4b74-973e-2932da3d98ef None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.716761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d5199-9580-4ccf-9084-d5c47a123697 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.749019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.749654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.749922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.750188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.757660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.759402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.760037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.760647 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.761079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.761359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.762515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:42.762820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.764237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:42.771242 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:42.772196 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 296/591] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:42.773552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.783279 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 296/592] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:42.784609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.854029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.879352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e3ec3-dd50-4a4a-9871-16d359f2566c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.910277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ff1f9-8faf-4e4e-8386-c96428885686 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:42.918336 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.918982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.919282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.919553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.920812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:42.921119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.923070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.947449 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 297/593] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:42.948617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:42.959969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:42.960594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:42.960874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:42.961143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:42.961879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:42.962486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:42.964240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:42.967671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:42.993517 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 297/594] 10.222.0.23 () {56 vars in 1139 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:42.994418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.068946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fcac6-d10e-4b04-8c6c-567144b163f7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.112170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e403910-6e9c-492d-bd65-57351e2c7d07 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.116619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.117391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.117765 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.118271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.120928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:43.129878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.131862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:43.146396 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:43.147347 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:43.148249 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:43.148676 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 298/595] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:43.149755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.153068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.153705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.153979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.154244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.168347 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.169803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:43.187788 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:43.188772 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:43.189845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:43.189845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:43.189845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:43.189845 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 298/596] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:43.190311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.247652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:43.270782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11387ce2-8953-4941-b35b-3144c138df7b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.305590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.306219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.306492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.306756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.308172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850b3d3-9173-4930-bc93-ab94cbae940d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.317486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.318921 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:43.334158 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:43.335245 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:43.336764 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:43.336764 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:43.336764 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:43.336764 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 299/597] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:43.336764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.350995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.351971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.352298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.352575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.353277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:43.353589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.355081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.369938 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 299/598] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:43.370697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.385927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:43.443918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da40d03-af43-43c2-b61d-b20a93f17776 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.479510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc329d-7dbd-43f7-867a-6d1c4f526f29 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.482244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.483121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.483619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.484077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.484933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:43.485425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.487311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:43.488309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.502413 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 300/599] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:43.503675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.521808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.522433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.522745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.523028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.523714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:43.524062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.525967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.543340 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 300/600] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:43.545320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.629461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bbdb6-84e7-4a2b-a1bf-cef06aaa752c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.639253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:43.674750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.675549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.675824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.676141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/8388bedc588f4f5f94f1e732d46e83b7/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.677177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4c39e-fa52-494e-be0e-e70639e274b2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.695900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=8388bedc588f4f5f94f1e732d46e83b7, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.697998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.715810 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 301/601] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:42:43 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/8388bedc588f4f5f94f1e732d46e83b7/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:43.716956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.717373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.723483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.724036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.724523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.734582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.736470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:43.751729 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:43.752981 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:43.754041 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:43.756898 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 301/602] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:43.763427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.797943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:43.890768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e152d102-23b2-4234-8dfc-0fe08425c015 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:43.936495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:43.937938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:43.938567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:43.938991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:43.939913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:43.940452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:43.942120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:43.958226 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 302/603] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:43.964738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:43.992785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b975c7-c752-4e0b-b068-e8c072cb1f79 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:44.031662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:44.032255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.032619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.032978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.040883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:44.042444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-36d59cf7-26e3-4ee0-a458-acbfaa92c67f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:44.055064 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:44.056067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:44.057162 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:44.058417 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 302/604] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:44.060149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787898fc-b7b2-405c-a53b-7df614a09250 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.060426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787898fc-b7b2-405c-a53b-7df614a09250 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.060690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787898fc-b7b2-405c-a53b-7df614a09250 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.062178 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 303/605] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:44.069501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053499f-0a59-4914-ac85-6eb7e1b4e76a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.070560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053499f-0a59-4914-ac85-6eb7e1b4e76a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.071002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053499f-0a59-4914-ac85-6eb7e1b4e76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.076893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:44.092463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1053499f-0a59-4914-ac85-6eb7e1b4e76a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:44.093886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1053499f-0a59-4914-ac85-6eb7e1b4e76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:44.102044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a03490-832b-4688-9472-7c1061d53d7c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:44.107035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:44.147922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:44.148707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.149073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.149432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/7d71b946e7d64a50888cfcc337d688a5/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.170126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=7d71b946e7d64a50888cfcc337d688a5, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:44.171681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:44.187980 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 304/606] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:44.195228 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 303/607] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:42:43 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/7d71b946e7d64a50888cfcc337d688a5/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:44.195228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:44.308017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c75bd-974a-4570-abfd-3ea08d2391fe None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.308380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c75bd-974a-4570-abfd-3ea08d2391fe None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.308568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c75bd-974a-4570-abfd-3ea08d2391fe None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.310502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7910-d6a5-4531-b75f-f5c4290c88fd None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:44.310781 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 305/608] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:44.315924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.316205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.316471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.351565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:44.352810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:44.353985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:44.354859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.355431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.356060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.357037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:44.357544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:44.359252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:44.367331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:44.373459 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 304/609] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:44.384134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:44.473108 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 306/610] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:44.512653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6446e8-b2c4-4359-9dc4-fa97d7fc8c20 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:44.519301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:44.555260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:44.556828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.557434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.557907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/341595ae0d9b4431b4417edda29f03ff/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.577976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=341595ae0d9b4431b4417edda29f03ff, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:44.580129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:44.617114 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 305/611] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:42:44 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/341595ae0d9b4431b4417edda29f03ff/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:44.619620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:44.658700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4359a531-c53d-4618-8bc9-ecf7860260f7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:44.722325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:44.723010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.723369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.723572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.740146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:44.744949 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:44.746086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:44.747103 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 307/612] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:44.753993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:44.834617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:44.880623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:44.917398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e562e2-ce0b-47c8-9843-fb91a15f24e3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:44.961191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:44.961801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:44.962072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:44.962333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:44.963310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:44.963608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:44.965220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:44.975339 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 306/613] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:44.983909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.030211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d034a-6ebc-426e-ac86-512a8605f3c7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.099532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.101426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.101779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.103026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.108244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:45.131824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:45.136623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:45.137743 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:45.138814 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 308/614] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:45.148204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.149079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12022147-0599-4bee-b551-6c60590a1020 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.159260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:45.185863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.186704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.187170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.187769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.199851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:45.201547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:45.216438 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:45.217645 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:45.218761 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:45.218761 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:45.218761 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:45.220201 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 307/615] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:45.226762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.351066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31819a25-3525-4feb-9232-2a169fe06780 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.399049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.401366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.402939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.403475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.404429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:45.405079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:45.408149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:45.422082 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 309/616] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:45.424617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5088c9-2416-4741-962e-b3efe3716e20 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.427978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.479215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.480342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.481028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.481552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.482409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:45.484432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:45.486248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:45.500181 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 308/617] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:45.505986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.540656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:45.560947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:45.615667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a8e75-4c5d-4647-98c0-aec172db236e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.650779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.651617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.651992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.652360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.663776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:45.665374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:45.680201 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:45.681294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:45.682502 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:45.682502 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:45.682502 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:45.682502 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 310/618] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:45.687910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.721328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a1f6d-3380-4690-8585-54166593a082 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.784284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.786966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.787289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.787569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.806259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:45.807770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:45.830485 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:45.831842 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:45.833241 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 309/619] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:45.839924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:45.871302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0e138-3a43-4a7d-bc28-55f7fa7aca7f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:45.885000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:45.934836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:45.935485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:45.935759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:45.936025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:45.937675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:45.937985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:45.939395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:45.963215 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 311/620] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:45.964325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:45.968498 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.017304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2682c8-a8e8-4220-a36e-9342c688b619 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.077285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e53cc-56a7-4e20-a6f1-07e4d0f79a87 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.114867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.115530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.115806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.116076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.119891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.120513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.120792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.121057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.121793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:46.122090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.123555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.123683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.125001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:46.139014 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:46.140052 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:46.141028 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:46.142297 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 312/621] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:46.149747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.155221 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 310/622] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:46.161815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.195383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:46.317200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4359-71d7-4a41-945d-89e29c7c5184 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.339266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17263a22-9774-4b85-ae68-56d3510f4c51 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.355566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.356236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.356443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.356711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.357785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:46.357895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.359316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87813b53-6f2c-42f4-b054-67d429d487af None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.378804 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 313/623] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:46.380764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348f7b8-8e51-4d54-8404-bb55b52b96ee None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.381047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348f7b8-8e51-4d54-8404-bb55b52b96ee None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.381365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348f7b8-8e51-4d54-8404-bb55b52b96ee None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.381787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.382619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.382861 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 314/624] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:46.383380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.383904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.384301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348f7b8-8e51-4d54-8404-bb55b52b96ee None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.393914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:46.468827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.471573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.492883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:46.507756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348f7b8-8e51-4d54-8404-bb55b52b96ee None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.541876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.542404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.542745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.543010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.553762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.555496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e3b31698-7bff-4dd6-8893-82b553980a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:46.574896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:46.575968 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:46.576911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:46.576911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:46.576911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:46.580493 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 315/625] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:46.582840 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.583609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.584196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.591311 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 311/626] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:46.592202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.608584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:46.610856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:46.638783 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:46.706937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a218b2-5bd3-4ec2-b44a-f836a49a70a4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.710917 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 316/627] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:46.712057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.744035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.744658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.744931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.745218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.746075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:46.746391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.747965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab9c87-dc18-42cf-aa3f-8a1d6d69e1da None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.759857 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 312/628] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:46.761053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389229a4-a281-438b-b66c-9c520db95f69 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.761330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389229a4-a281-438b-b66c-9c520db95f69 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.761597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389229a4-a281-438b-b66c-9c520db95f69 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.763112 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 313/629] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:46.772488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.772760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.773035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.796104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:46.798544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:46.799979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:46.840468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ff416-01ad-4e10-a655-ef7d30e9d273 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:46.883059 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 314/630] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:46.883059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:46.892136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:46.892781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:46.893051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:46.893493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:46.917024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:46.920960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:46.922449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:46.953544 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:46.954565 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:46.955519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:46.955519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:46.955519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:46.955519 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 317/631] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:46.964100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.008149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904e577-40bc-4145-94e7-f8e9e43d9ba2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.052935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.053636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.053915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.054188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.064672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.066160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.078830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa12df-d084-4753-9f9e-3b46d67a44cb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:47.083258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:47.084171 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:47.085069 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 315/632] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:47.086508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.109556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.123280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.124132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.124606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.125102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.126078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:47.126594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.128299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.141429 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 318/633] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:47.144821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.198413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5d75c-7c68-415b-b092-437da97d57ed None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.201449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.237825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.238483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.238762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.239036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.251150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:47.256055 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:47.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:47.257893 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 316/634] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:47.258648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.265960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a870adb-ce5d-49f6-ad4d-5ebc3043a41d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.302018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.302677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.302897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.303139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.304056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:47.304175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.305533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.322713 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 319/635] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:47.322713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.367719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ecc95-e46b-4085-ad0e-c3d058ad6d2f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.367919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.401009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.401578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.401850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.402115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.408633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.410102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:47.420959 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:47.422062 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 317/636] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:47.423327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.423896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505f1a2-2ff5-4a16-9896-ef65a3a73ae4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.433561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.459985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.460607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.460874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.461137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.471068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:47.477114 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:47.478265 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 320/637] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:47.479263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.528620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb545151-d55d-4a41-b779-baa6a3364820 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.563401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.564003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.564268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.564532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.565318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:47.565612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.566973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ba60a-edcd-4012-b717-c63a2e7707ec None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.574967 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 318/638] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:47.576102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fef788-e989-4409-bff0-3edd7ec4b754 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.576381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fef788-e989-4409-bff0-3edd7ec4b754 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.576645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fef788-e989-4409-bff0-3edd7ec4b754 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.577886 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 319/639] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:47.578564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fef788-e989-4409-bff0-3edd7ec4b754 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.582607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e39659-192d-4df4-925c-7ad5a30a6edc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.584410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.617583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.618230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.618509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.618774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/512080b44c7542439333bd8b5f941496/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.632830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=512080b44c7542439333bd8b5f941496, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.634264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.646943 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 321/640] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:42:47 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/512080b44c7542439333bd8b5f941496/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:42:47.647957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.659873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.682725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fef788-e989-4409-bff0-3edd7ec4b754 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.725145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.725976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.726346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.726678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.727423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:47.727728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.729233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.742265 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 320/641] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:47.748320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.762860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8caf3b4-4f84-45dd-89e5-9a018733eb0c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.800092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.801030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.801297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.801567 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.802247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:47.802541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.803974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02be17-d236-48a8-9edb-ddf9cd25152e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.813927 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 322/642] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:47.816141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.816417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.816680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.837695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:47.838905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:47.848761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.870602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46557a6-0231-4b59-a78c-aa8728c5167d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:47.901072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:47.908704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:47.909345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:47.909619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:47.909941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:47.912445 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 323/643] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:47.916020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:47.928737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:47.930431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:47.946607 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:47.947882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:47.949017 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:47.949017 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:47.949017 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:47.951106 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 321/644] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:47.952091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.038785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef0053-6892-449a-af14-c36e07adbf06 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.065846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3bf63-555d-434c-9112-8d2d5e7159fe None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.079219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.080318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.081253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.081680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.082812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=c150143af9144547ab3dc5a5d318d45f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:48.083330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.085498 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.093708 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 324/645] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/role_assignments?user.id=c150143af9144547ab3dc5a5d318d45f&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:48.094661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.124926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.125360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:48.126177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.126177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.126653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.165024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.166679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:48.189454 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:48.190754 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:48.192097 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:48.192097 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:48.192097 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:48.196573 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 322/646] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:48.203329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.244045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:48.250981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8a5fd-cdc9-45e7-963a-ce0803b9f064 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.289985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.290611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.290885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.291152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.291864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:48.292186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.293818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.313786 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 325/647] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:48.314772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.333546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa59a-2ff5-41f4-9b60-c25d91fcd598 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.381928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.382691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.382971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.383683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.383961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:48.384235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.385627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.398419 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 323/648] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:48.399310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.425274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:48.467467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1cdc0b-198c-4e5a-b9ba-ca9bff82a554 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.505149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.506136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.506585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.507033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.514271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:48.519287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:48.521907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb9311-fc84-49bd-8ae3-d1be05c5e06b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:48.525527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:48.526694 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:48.527795 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:48.530099 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 326/649] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:48.531072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.569690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.570606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.571025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.571485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.582092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.584356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:48.599670 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:48.600751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 11 00:42:48.601720 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:48.604510 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 324/650] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:48.605446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.652727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acf31e-a12b-4de0-a8f0-c2f4ae25d1aa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.687740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:48.699291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.699931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.700201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.700462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.717292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.719394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:48.735375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:48.736387 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:48.737416 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 327/651] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:48.738048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.743636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b2073-d080-4824-b07b-040ebea2fc45 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.792699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.793387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.793658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.793918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.794574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:48.794891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.796326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.809846 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 325/652] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:48.810716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.862755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006ef3a-1f5f-4b56-afa0-42ce9ebe62d8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.863484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:48.907008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.907815 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.908184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.908546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.909424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:48.909828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.911384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.916329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3376d1-5032-4ddf-b3d9-693f2091e730 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:48.929573 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 328/653] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:48.930587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.951150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:48.952046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:48.952465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:48.952864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:48.953724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:48.954157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:48.955816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:48.973838 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 326/654] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:48.974975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:48.985775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.043796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cd496b-7161-4b52-8ece-89b7f7385f54 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.071540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.081472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.082263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.082683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.083048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.093644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ead41-ef3e-4b90-bb06-342b773c71a1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.094086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.095789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:49.112013 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:49.113219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:49.114317 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:49.114317 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:49.114317 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:49.115305 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 329/655] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:49.116150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.135281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.136118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.136593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.137016 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.215467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.218231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.229513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.234015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b8eaa0-e6b9-4d9c-a242-576b99555380 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.270298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.271074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.271494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.271853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.319141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.347762 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 327/656] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:49.348733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.360294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.366767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.452491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bd5bd-10d7-46f7-afab-b288736b5a47 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.476314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.482183 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 330/657] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:49.484110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.494702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.495669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.495831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.496258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.497000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:49.497399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.498881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.511043 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 328/658] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:49.512076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.547136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.596293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f250cff-7419-4ca3-9d07-c5b163f14e29 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.616987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee74d4a-6736-4469-b343-126a87ff4505 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.633271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.634269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.634845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.635320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.646682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.648591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.652880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.653564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.653839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.654102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.664664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:49.666259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:49.667190 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:49.668099 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:49.668099 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:49.668099 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:49.668268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.668892 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 331/659] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:49.669571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:49.688593 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:49.689597 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:49.690740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:49.690740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:49.690740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:49.696987 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 329/660] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:49.696987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.734052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.792362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9546c-7a03-4737-89e4-83ff7bb2206e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.793118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:49.818830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec54924-5546-46cb-b920-37842252c081 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.839658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.840739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.841267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.841725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.850975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.852847 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.858325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:49.858940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:49.859236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:49.859510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:49.860156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:49.860454 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:49.861843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:49.866576 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:49.867554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 11 00:42:49.868603 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:49.871500 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 332/661] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:49.872682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.875010 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 330/662] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:49.880058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:49.977552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fad06-c79f-4093-9276-cd0592b35b74 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.983500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22182626-60f5-4ee0-836a-245599ce653d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:49.988870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.013482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.014076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.014344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.014609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.015272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:50.015569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.016951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.018622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.019522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.019522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.019817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.026488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.027930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.034198 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 333/663] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:50.035082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4f9cdcfe-e8ff-477c-9b5d-6e1ce4ef5f65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:50.039398 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:50.040258 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 331/664] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2a47-ce4e-4f02-bca8-d87e841f5cd8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.048324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2a47-ce4e-4f02-bca8-d87e841f5cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.048943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc2a47-ce4e-4f02-bca8-d87e841f5cd8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.048943 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 332/665] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:50.048943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2a47-ce4e-4f02-bca8-d87e841f5cd8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.058575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.142851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fedf5-a513-41d5-90f5-63ddf679e27a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.161545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc2a47-ce4e-4f02-bca8-d87e841f5cd8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.187611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.188216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.188485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.188742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.189460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:50.189752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.191269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.197474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.198107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.198377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.198649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.205627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.207082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.214683 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 334/666] 10.222.0.23 () {56 vars in 1139 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:50.214683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e7fb86f7-2d57-41dd-aa10-1d5b998e1d8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:50.218687 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:50.219980 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 333/667] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.226102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.226725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.241491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:50.242601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:50.246389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.310054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.318982 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 334/668] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:50.319857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.333113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e804f3-4c25-4eaf-9513-4fc66f3e576c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.370828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.371503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.371774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.372044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.372779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:50.373070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.374696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.388574 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 335/669] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:50.389554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.464881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec41bb4-b845-481e-b075-c45e876dc102 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.499617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.505566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c6165-6484-43b2-bedf-a235a610f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.506179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c6165-6484-43b2-bedf-a235a610f420 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.506448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c6165-6484-43b2-bedf-a235a610f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.506708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c6165-6484-43b2-bedf-a235a610f420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.513486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1a786-8e11-40ca-9d5c-6f7e2a60d248 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.550274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.551357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.551634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.551895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.552585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:50.552887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.554317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.572389 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 336/670] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:50.572389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.575120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.579352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c6165-6484-43b2-bedf-a235a610f420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.580927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c6165-6484-43b2-bedf-a235a610f420 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.674020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54df2-9f1c-40e8-a5ac-be24759d6420 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.696809 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 335/671] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:50.698578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4d88-53c2-4e31-aef3-1a1fe739556d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.699189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4d88-53c2-4e31-aef3-1a1fe739556d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.699623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b4d88-53c2-4e31-aef3-1a1fe739556d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.701171 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 336/672] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:50.702045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b4d88-53c2-4e31-aef3-1a1fe739556d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.713767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.714512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.714794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.715043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.715807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=8388bedc588f4f5f94f1e732d46e83b7) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:50.716097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.717523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.723873 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 337/673] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/role_assignments?user.id=8388bedc588f4f5f94f1e732d46e83b7&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:50.724730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.740194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.831879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:50.842354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b4d88-53c2-4e31-aef3-1a1fe739556d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.843698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca88c3f-b857-44e3-a5f7-09ed853b20c5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:50.883236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.884041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.884317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.884585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.887784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:50.888408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.888679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.888943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.889682 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=7d71b946e7d64a50888cfcc337d688a5) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:50.889989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:50.891481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d6085-ef77-4616-88f8-3911fb9a9cab None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:50.896936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:50.902183 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:50.903272 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:50.904151 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:50.908933 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 338/674] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:50.909359 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 337/675] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/role_assignments?user.id=7d71b946e7d64a50888cfcc337d688a5&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:50.909848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:50.911492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:50.912502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:50.914005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:50.947792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:50.949024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:51.058820 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 338/676] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:51.060374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.115669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.132720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf15d4a-3808-445e-84c4-758c6b52457e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.180739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.180739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.180739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.180739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.195220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.197449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.207601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:51.217794 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:51.218793 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:51.220103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:51.220103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:51.220103 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:51.220103 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 339/677] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:51.223078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7180bb-1cf2-46ad-ab88-84a875d058f2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.224530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.265187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.266098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.266469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.266894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.267703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:51.268155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.269963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.279015 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 339/678] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:51.284796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.340696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206c3e4-1314-407f-a496-c1c099bfe2fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.365415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.379625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.380148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.380416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.380675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.381334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:51.381601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.382966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.393278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64dba4-4604-45cd-bd50-7407241dc4be None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.398063 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 340/679] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:51.398962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.435589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.436626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.437149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.437591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.450241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.452352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:51.469539 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:51.470563 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:51.471558 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:51.471558 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:51.471558 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:51.474504 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 340/680] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:51.475616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.480112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.511011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c5689-cd53-4c01-af61-b3d6a0644a13 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.548631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.550186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.550487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.550758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.562866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:51.567823 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:51.568788 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 341/681] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:51.570229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.608270 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ba637-6c3b-436e-a18d-ff245a90a285 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.627345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.649986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.650855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.651326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.651734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.660698 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.663141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:51.677234 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:51.678480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:51.679625 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:51.687463 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 341/682] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:51.687463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.700597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dd4ff-b23a-4b38-bb96-ad5bd87614ce None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.746002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.747276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.748175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.748902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.749993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:51.750966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.753119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.753965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.776163 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 342/683] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:51.777193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.804963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c135b9e-bcff-4a67-834f-979b9eb43fd1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.844746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.845724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.846228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.846804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.847812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:51.848307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.850097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.862815 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 342/684] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:51.864257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.883383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea1309-6300-476f-a943-4fd5cc757b99 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:51.886585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:51.920772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:51.921294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:51.921567 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:51.921832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:51.922590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:51.922889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:51.924639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:51.941916 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 343/685] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:51.941916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:51.994657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f5c6b7-e1e0-41c8-95f6-8bade7eee4db None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.026209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.037843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.038535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.038823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.039084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.047395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.048979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:52.061479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:52.062457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 343/686] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:52.064089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.074444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac4405-71da-4cd5-9723-65bd6abf48c0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.124777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.125409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.125677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.125936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.137807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.142334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:52.162055 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:52.163157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:52.164155 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:52.164155 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:52.164155 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:52.171869 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 344/687] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:52.171869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.188797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a084b28-6720-49e7-a626-b672eb888799 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.190003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.229079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.230111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.230384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.230794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.231682 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=341595ae0d9b4431b4417edda29f03ff) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:52.231983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.236330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.255581 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 344/688] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/role_assignments?user.id=341595ae0d9b4431b4417edda29f03ff&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:52.256470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.329061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376e493-a10e-482c-a09e-c842850e65ed None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.360834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.383785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.385129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.385620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.388431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.389284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:52.389720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.391265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.429305 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 345/689] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:52.433750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.465587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748ce9-4780-42e4-a8f9-6d825c7cf3a2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.503759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.504507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.504868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.505225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.506069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:52.506461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.508051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.521916 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 345/690] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:52.527890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.537157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.567440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca9cad-b5c7-4683-a648-fa5eadcce1b7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.607675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.608285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.608599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.608864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.636124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee0bb-d5cb-40fa-bc8c-670cae5d945d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.648298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.673026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.673833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.674143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.674643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.682917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.683169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.684942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.685333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:52.697705 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:52.698990 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:52.699989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:52.700522 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 346/691] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:52.706114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.764942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.794025 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 346/692] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:52.799144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.819093 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceedcd-6271-4876-83c6-a46895fb0fa2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.865821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.866848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.867368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.867842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.868773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:52.869324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.870961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.899888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccd1ea-6f92-4011-a840-76d20858fb89 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:52.901929 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 347/693] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:52.909243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:52.941478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:52.942021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:52.942294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:52.942557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:52.960278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:52.961597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:52.963078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:52.978899 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:52.979957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:52.980886 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:52.980886 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:52.980886 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:52.980886 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 347/694] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:52.985462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:53.030480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:53.104895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3434890-bda1-48cd-8918-93bd9bb8d9b5 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:53.152007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178a0b6-f395-4f9a-aae1-200aa30474f3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:53.153342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38aed2-8d30-435f-9db2-3e553b8138be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:53.156408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38aed2-8d30-435f-9db2-3e553b8138be None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.156408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38aed2-8d30-435f-9db2-3e553b8138be None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.156408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38aed2-8d30-435f-9db2-3e553b8138be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.222115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:53.223154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.223659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.225424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.226323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:53.226852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:53.229402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:53.253693 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 348/695] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:53.260772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:53.272099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38aed2-8d30-435f-9db2-3e553b8138be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:53.275722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd38aed2-8d30-435f-9db2-3e553b8138be None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:53.327544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:53.398863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:53.415116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd4a4b-d9ed-4ac4-9958-14fa1732f87a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:53.423271 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 348/696] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:53.424740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59f89c-9d18-4108-baa7-402e8656884e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.425169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59f89c-9d18-4108-baa7-402e8656884e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.425532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59f89c-9d18-4108-baa7-402e8656884e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.427153 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 349/697] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:53.430023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59f89c-9d18-4108-baa7-402e8656884e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:53.464842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:53.465536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.465893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.466391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.475231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:53.476652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3854ae33-d36d-4b61-a395-65aca12184f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:53.491390 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:53.492375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:53.493323 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 349/698] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:53.493846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ae63-8dd0-4173-81b0-6af9f9af71b8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.493924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ae63-8dd0-4173-81b0-6af9f9af71b8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.494574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ae63-8dd0-4173-81b0-6af9f9af71b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.516135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d2c6ae63-8dd0-4173-81b0-6af9f9af71b8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:53.517320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d2c6ae63-8dd0-4173-81b0-6af9f9af71b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:53.546299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59f89c-9d18-4108-baa7-402e8656884e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:53.574294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:53.583465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:53.584115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.584372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.584639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.595410 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 350/699] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:53.595410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fc2ee-daba-4bff-9d6f-7711d1c24c06 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.595410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fc2ee-daba-4bff-9d6f-7711d1c24c06 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.595410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fc2ee-daba-4bff-9d6f-7711d1c24c06 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.595410 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 351/700] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:53.595410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fc2ee-daba-4bff-9d6f-7711d1c24c06 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:53.610637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:53.614661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7d918187-332b-4dc4-9059-d9b7e6da9b3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:53.641267 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:53.642537 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:53.643535 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:53.643535 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:53.643535 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:53.643535 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 350/701] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:53.644339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.647200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.647200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.666343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:53.668084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:53.705841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:53.736288 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 351/702] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:53.737368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:53.745060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fc2ee-daba-4bff-9d6f-7711d1c24c06 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:53.805295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:53.805911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.806614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.806886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.807615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:53.808271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:53.810401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:53.834938 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 352/703] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:53.840913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:53.903788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b1fac-a006-4d3a-b416-85ff090d9aa9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:53.917010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:53.954785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:53.955436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:53.955765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:53.955998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:53.956648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:53.956943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:53.958309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:53.976596 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 352/704] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:53.976596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.004180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bd7bf-63c5-4e88-8907-4195d4d2800b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.052227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.053234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.053737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.054725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.055726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=512080b44c7542439333bd8b5f941496) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:54.056702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.059388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.063823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:54.075205 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 353/705] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:53 2022] GET /identity/v3/role_assignments?user.id=512080b44c7542439333bd8b5f941496&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:42:54.075205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.088824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de216030-dd19-4931-9524-4a7c9d1d131d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.133771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.134423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.134694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.135638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.149225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.150702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:54.167769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:54.168820 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:54.169732 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:54.169732 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:54.169732 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:54.169732 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 353/706] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:54.175201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.194167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd809e35-7b89-4faf-9ce2-47a59c552957 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.201855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:54.232925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.233632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.233905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.234174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.243301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.244780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:42:54.258866 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:54.260047 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:54.261154 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 354/707] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:54.261852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.310601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853835d6-0470-4270-bb33-2b4626c62363 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.332612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:54.352457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.353096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.353364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.353618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.354274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.356035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:54.370457 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:54.371592 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:54.372475 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 354/708] 10.222.0.23 () {56 vars in 1129 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:54.373343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.389077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee7f71-edea-45f4-9ccf-0e92275c489a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.433635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.435325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.435770 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.436201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.437097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:54.438088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.440068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.459129 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 355/709] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:54.460346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.472861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:54.490021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a320b9-4477-42dc-8be5-b273b0a1001a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.533706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.534863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.535341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.536037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.537307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:54.538014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.539684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.557043 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 355/710] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:54.558481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.587385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d31f-f268-4e7f-bde7-5d8c7a5e8db0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.615544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:54.635483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.636554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.636965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.637603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.639287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:54.639735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.641248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.652084 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 356/711] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:54.652291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.676459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc171-a113-43d3-9adc-778041c52eb3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.715600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.716266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.716538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.716803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.717581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.719241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.743526 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 356/712] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:42:54 2022] POST /identity/v3/users => generated 254 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:54.744384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:54.773746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a9b610-0682-41f8-ba5d-51ea6317ddb7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.812025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:54.820998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.821727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.822085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.822383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.823110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=c150143af9144547ab3dc5a5d318d45f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:54.823433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.824857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e93c8-9e9d-43e1-a926-dbacc1025ed7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.840118 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 357/713] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/role_assignments?user.id=c150143af9144547ab3dc5a5d318d45f&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:54.868172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86af1-68c4-439b-b78d-840d65903a0e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:54.903152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c521-d518-4567-8423-5bdcdef59e4a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.903477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c521-d518-4567-8423-5bdcdef59e4a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.903724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4c521-d518-4567-8423-5bdcdef59e4a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.905209 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 358/714] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:42:54.911329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.911639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.911871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.919935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:54.920575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:54.920849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:54.921120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:54.921764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:54.922064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:54.923513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:54.939335 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 357/715] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Sep 11 00:42:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:54.944932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:54.946190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:55.025916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:55.067079 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 359/716] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:55.337559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:55.404725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:55.440318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a766ca-2ee3-4510-8f33-471d26551c2e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:55.474485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:55.475333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:55.475605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:55.475868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] PATH_INFO: `/v3/services/network` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:55.476446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:55.477794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:55.487262 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:55.488314 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:42:55.489190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:55.493143 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 358/717] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:55 2022] GET /identity/v3/services/network => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:55.494519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:55.601725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c49e5f-06be-44a0-beaf-d486e221be1e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:55.640046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:55.640821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:55.640821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:55.641106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:55.641689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] RBAC: Adding query filter params (name=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:55.641984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:55.643747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982ccfc-73e1-455d-b018-f7cf320fd711 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:55.651396 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 360/718] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:42:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:55.654196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:55.767422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b190-9325-46e6-b289-790df7325f90 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:55.796193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:55.803534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:55.804140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:55.804407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:55.804673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:55.805292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] RBAC: Adding query filter params (type=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:55.805583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:55.806944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcce119-4686-47df-95ee-f4e3413ca711 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:55.817123 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 359/719] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:42:55 2022] GET /identity/v3/services?type=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:56.808351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb08a90-437c-40d3-8cdc-0c924b971881 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:56.809389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb08a90-437c-40d3-8cdc-0c924b971881 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:56.809389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb08a90-437c-40d3-8cdc-0c924b971881 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:56.810785 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 361/720] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:56.816196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:56.816678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:56.816854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:56.835366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:56.836574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:56.908534 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 360/721] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:57.248198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb08a90-437c-40d3-8cdc-0c924b971881 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:57.302175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:57.378518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb08a90-437c-40d3-8cdc-0c924b971881 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:57.423111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97626271-4ab3-4e1c-837c-6b7fa0e072e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:57.423701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97626271-4ab3-4e1c-837c-6b7fa0e072e0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.423975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97626271-4ab3-4e1c-837c-6b7fa0e072e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.424299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97626271-4ab3-4e1c-837c-6b7fa0e072e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.435649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97626271-4ab3-4e1c-837c-6b7fa0e072e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-97626271-4ab3-4e1c-837c-6b7fa0e072e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:57.440440 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:57.441380 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:57.442228 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 362/722] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:57.449303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:57.512628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6e29f-85d3-4837-8ad0-c5839c1bfc67 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.512963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6e29f-85d3-4837-8ad0-c5839c1bfc67 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.513239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6e29f-85d3-4837-8ad0-c5839c1bfc67 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.515476 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 363/723] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:57.521469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0a343-1931-43bf-8027-e0976f49dbaf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.521750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0a343-1931-43bf-8027-e0976f49dbaf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.522022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0a343-1931-43bf-8027-e0976f49dbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.560296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7ac0a343-1931-43bf-8027-e0976f49dbaf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:57.561621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ac0a343-1931-43bf-8027-e0976f49dbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:57.576536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd254f7e-572e-4d6d-99a5-9d73cf5d4e0c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:57.603161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:57.616576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:57.617554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.618063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.618543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.619439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:57.619929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:57.621476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:57.630605 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 361/724] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:57.636112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:57.675137 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 364/725] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 11 00:42:57.675137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900721-bea0-4145-9e92-0e1368ade200 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.675137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900721-bea0-4145-9e92-0e1368ade200 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.675137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09900721-bea0-4145-9e92-0e1368ade200 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.675137 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 365/726] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:57.680107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.680381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.680651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.715417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:57.716620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:57.769150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:57.809800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d9571-5964-4b40-a370-4a2efbcfcda4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:57.824005 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 366/727] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:57.866081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:57.866759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:57.867026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:57.867351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:57.886839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:57.888342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:57.909923 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:57.910956 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:57.912209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:57.912209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:57.912209 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:57.912209 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 362/728] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:57.920753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.026769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504a916-f093-4144-b85d-07c819c7ac26 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.044843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.069461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.070077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.070345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.070605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.071266 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:58.071775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.072912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab65884-ff7d-4ecd-81b6-1530b32ac00a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.086557 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 367/729] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:58.087821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431763c-6828-4362-a539-5bcd24d9605a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.088097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431763c-6828-4362-a539-5bcd24d9605a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.088362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431763c-6828-4362-a539-5bcd24d9605a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.089840 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 368/730] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:58.095283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431763c-6828-4362-a539-5bcd24d9605a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.109221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.120446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.179458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb634f9-d99d-4263-8bdc-81913a2177ec None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.207949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431763c-6828-4362-a539-5bcd24d9605a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.213102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90794406-0dec-4128-8170-483c453dc6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.213733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90794406-0dec-4128-8170-483c453dc6ff None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.214006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90794406-0dec-4128-8170-483c453dc6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.214273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90794406-0dec-4128-8170-483c453dc6ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.225791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90794406-0dec-4128-8170-483c453dc6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-90794406-0dec-4128-8170-483c453dc6ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:58.231536 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:58.232480 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:42:58.233371 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 363/731] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:58.234080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.234080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.234805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.247028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.247691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.247967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.248233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.253338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:58.254714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:58.255977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.257433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:58.269152 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:58.270157 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 369/732] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:42:58.271092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.336785 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 364/733] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:58.347418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.372965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.380584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.394654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b384297-a004-40c3-8ed3-8ede5e90ce0d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.432237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.432892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.433218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.433488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.447000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:42:58.452991 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:58.454009 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:58.454879 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 370/734] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:58.455698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.477854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a525b-cd37-466e-8e3d-fa57f143e11e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.527217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.528084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.528358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.528625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.529530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:58.529828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.531879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.543215 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 365/735] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:58.544702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.574706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221ce7-5f26-40f9-b63b-2faa75ec7d02 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.611645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.612270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.612546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.612803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.613520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:58.613808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.615221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.630199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.634833 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 371/736] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:58.636234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.648614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.658108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77799aa8-a146-4767-8848-f007e65a62d3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.690529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.691130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.691429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.691713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.692359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:58.692654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.694019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316754b-5845-4468-8c86-bb9820774a77 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.701618 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 366/737] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:58.702737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8fb5-56b2-4e33-8573-5afc842f6cd7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.703015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8fb5-56b2-4e33-8573-5afc842f6cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.703315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8fb5-56b2-4e33-8573-5afc842f6cd7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.704490 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 367/738] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:42:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:42:58.705170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8fb5-56b2-4e33-8573-5afc842f6cd7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.741119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a23c3-80cb-44e5-a22b-c8f63712feac None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.773473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373421ae-3292-4917-9920-136951b80e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.774102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373421ae-3292-4917-9920-136951b80e4d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.774380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373421ae-3292-4917-9920-136951b80e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.774645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373421ae-3292-4917-9920-136951b80e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.784443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373421ae-3292-4917-9920-136951b80e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.785876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373421ae-3292-4917-9920-136951b80e4d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-373421ae-3292-4917-9920-136951b80e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:58.799952 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:58.801060 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:58.802170 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:58.802170 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:58.802170 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:58.805307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8fb5-56b2-4e33-8573-5afc842f6cd7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.805760 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 372/739] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:58.806455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373421ae-3292-4917-9920-136951b80e4d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:58.840611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9d434-68a0-46b0-8b13-7bac6f1cfd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.841214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9d434-68a0-46b0-8b13-7bac6f1cfd01 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.841481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9d434-68a0-46b0-8b13-7bac6f1cfd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.841741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9d434-68a0-46b0-8b13-7bac6f1cfd01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.857922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.863649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:58.905313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373421ae-3292-4917-9920-136951b80e4d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:58.909382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9d434-68a0-46b0-8b13-7bac6f1cfd01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.910832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9d434-68a0-46b0-8b13-7bac6f1cfd01 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.940525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:58.941111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:58.941396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:58.941667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:58.942303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:58.943691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:58.960408 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 373/740] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:42:58 2022] POST /identity/v3/services => generated 251 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:58.961199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.025812 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 368/741] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 11 00:42:59.027124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.027448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.027714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.047149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:42:59.048432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:42:59.085101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b01c4-c1e7-44b4-a27c-bb9c86ada60c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.096982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.103410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.127027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.127647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.127920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.128186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.139485 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 369/742] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:42:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 11 00:42:59.139485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.145027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.146530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:59.162951 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:59.163954 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:59.164854 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:59.164854 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:59.164854 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:59.169673 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 374/743] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:59.169673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.253210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfbf95-1fe8-4314-9d6b-6347b21f304e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.289017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.289729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.290099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.290437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.291147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:59.291565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.293005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.295855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1aea12-104b-49a1-9609-8c485e462c9e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.305308 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 370/744] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:59.306279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.345916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.346701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.347041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.347455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.358577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.360171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.364537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:42:59.375384 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:59.376375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:42:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:42:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:42:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:59.381282 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 375/745] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:59.382353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.398711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.439994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001efbb-129c-42a5-904f-dcdabb021ad1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.477111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.477807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.478080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.478349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.479003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:59.479324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.481407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.489208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36a8a9-0c02-43da-a633-bd1a3995859e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.500081 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 371/746] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:59.500081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.532251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.532883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.533156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.533418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.541164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.542603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:42:59.554914 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:42:59.555911 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 376/747] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:42:59.556888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.632378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6d7be-3261-42e7-bd3d-465d18da21cb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.634596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.639839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.671610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.672219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.672393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843ef7d-1318-4a6e-9b5f-1f91937c7a28 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.672492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.672739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.673372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:59.673665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.675099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.687572 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 372/748] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:59.688174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.712492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.713197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.713547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.713883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.714601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.716063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.731474 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 377/749] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:42:59 2022] POST /identity/v3/services => generated 240 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:42:59.732287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.865759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5bc02-781d-4c7f-a7d1-0b37ac1f2d6d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.900684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:42:59.911450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.912495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.912977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.913451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:42:59.914444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:42:59.915899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:42:59.918470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:42:59.944327 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 378/750] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:42:59.945661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:42:59.949333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b03c46-468d-407a-b51b-473b05ffe01e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:42:59.992727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:42:59.993358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:42:59.993629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:42:59.993888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.002159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.003995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:00.016913 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:00.018204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:00.019505 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:00.022803 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 373/751] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:00.023629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.053760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.067705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60aa8a-93af-4fde-abd0-563a4e669196 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.104198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.105077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.105628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.106041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.113518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.115363 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:00.128428 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:00.129553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:00.130638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:00.133039 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 379/752] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:00.134135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.136414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beddee-b3af-4fe7-ad2c-dc44009c3704 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.190349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.192259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.192795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.193243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.193980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.207172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.208906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:00.229366 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:00.230545 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:00.231700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:00.231700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:00.231700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:00.233612 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 374/753] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:00.234760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.256269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471adca-0820-400c-a44d-d1541f06e3c3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.300687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.301552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.301895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.302230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.303372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:00.304002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.305675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.329569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.331161 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 380/754] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:00.332941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.360893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8f32b-8774-4c3d-bb66-405cb3405a82 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.402129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.402914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.403323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.403600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.404291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:00.404592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.406263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.425572 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 375/755] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:00.426190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.458919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccacd2b7-1fc5-4134-ac69-6e212eb2a22e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.462455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.497037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.497924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.498473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.498978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.499853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:00.500332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.501845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.517260 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 381/756] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:00.519010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.542188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05bfb2-505a-431f-93bc-a278009eb83a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.583402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.584091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.584368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.584629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.587894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.656078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bde24cb-9745-4ab9-9917-094d01a6def8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.659149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.660830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.700369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.700951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.701773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.702052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.702788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=512080b44c7542439333bd8b5f941496) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:00.703095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.705024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.715334 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 382/757] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/role_assignments?user.id=512080b44c7542439333bd8b5f941496&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:00.716453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.779487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.820155 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 376/758] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:00.825381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:00.900511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:00.911485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df135d-4edd-4c4d-871b-3f3374b4773a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:00.949096 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:00.949720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:00.949990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:00.950249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:00.958387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:00.959816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:00.972068 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:00.973015 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:00.973929 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 383/759] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:00.982402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:01.015021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf0b87-2809-425a-afb3-a43cd83666c3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:01.085259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:01.085895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.086171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.086435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.109765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:01.111234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:01.145106 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:01.146160 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:01.147077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:01.147077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:01.147077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:01.147077 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 377/760] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:01.158019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:01.172076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:01.205293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf514a-22ee-4767-b715-7491806511e7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:01.271114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:01.271853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.272224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.272583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.273369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:01.273768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:01.275456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93925aa-0e61-40e3-be35-047804dd9f9a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:01.292000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:01.293318 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 384/761] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:01.294901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d28a-c8bb-4746-aac1-2e0e0c779ed5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.295351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d28a-c8bb-4746-aac1-2e0e0c779ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.295741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126d28a-c8bb-4746-aac1-2e0e0c779ed5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.298536 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 385/762] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:01.300008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d28a-c8bb-4746-aac1-2e0e0c779ed5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:01.314416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75b748-ac0d-4563-b26e-e7d67c60495d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:01.360805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:01.361465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.362027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.362340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.363495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:01.364042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:01.365666 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b1a308-311a-44f5-b409-f9dc8a0dcb28 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:01.383277 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 378/763] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:01.384696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.385229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.385632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.403975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:01.405306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:01.418281 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126d28a-c8bb-4746-aac1-2e0e0c779ed5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:01.462034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:01.462843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.463200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.463571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/c150143af9144547ab3dc5a5d318d45f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.465615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=c150143af9144547ab3dc5a5d318d45f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:01.467164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:01.487347 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 386/764] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:01 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/c150143af9144547ab3dc5a5d318d45f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:43:01.488470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:01.494196 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 379/765] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:01.514726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:01.658647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1095aae1-4a9b-40af-b56b-8d591a9e2c11 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:01.710318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:01.710962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.711269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.711541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.718939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:01.720507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:01.732459 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:01.733381 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:01.734295 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:01.739223 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 387/766] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:01.739371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:01.823967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:01.836600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:01.866067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91850c8-24ec-401a-8b92-d94eaaed1ca5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:01.905862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:01.906879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:01.907388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:01.907848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:01.908857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:01.909365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:01.910993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:01.924984 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 380/767] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:01.931578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:02.020866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:02.041135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222b46c-4680-4fc3-b603-d98ac283928e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:02.080196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:02.081768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.082245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.082724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.083764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=8f67555ada824965b62e7026cfbab0bf) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:02.084545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:02.086234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:02.091455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:02.094150 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 381/768] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:01 2022] GET /identity/v3/role_assignments?user.id=8f67555ada824965b62e7026cfbab0bf&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:02.113527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c3974-b199-439f-aa00-65269bb8529c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:02.163072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:02.163072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.163072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.163072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.163441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:02.163441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b69b344c-04ec-4fb4-a242-50b72c96aae8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:02.192217 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:02.193303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:43:02.194144 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:02.198286 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 388/769] 10.222.0.23 () {56 vars in 1129 bytes} [Sun Sep 11 00:43:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:02.201541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:02.212029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254003-1abe-421b-ad00-af43898a59ff None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.212344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254003-1abe-421b-ad00-af43898a59ff None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.212573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254003-1abe-421b-ad00-af43898a59ff None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.214294 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 389/770] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:02.219706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.219992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.220251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.239559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:02.240769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:02.360868 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 390/771] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:02.364290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2118803-afc5-411f-ab35-991a9febb8ae None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:02.403795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:02.404916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.405418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.405928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.406748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:02.407265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:02.408893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:02.414456 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 382/772] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Sep 11 00:43:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:02.421932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:02.518513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:02.569389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596c838-56a1-4930-b387-8504d3d0bfa4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:02.621552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:02.622183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.622452 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.622712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.623395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:02.623689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:02.625118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:02.636960 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 391/773] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Sep 11 00:43:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:02.638659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:02.652475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:02.748885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:02.815335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5afef-d21c-4c2b-933e-38eef5fe1bf8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:02.826808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81d4f0-a8d9-4b65-9c7c-5f60e8e4970d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:02.868980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:02.877236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:02.878298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.878567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.878897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] PATH_INFO: `/v3/services/network` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.879510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:02.881393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:02.884830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:02.885442 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:02.885709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:02.885971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:02.886667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] RBAC: Adding query filter params (interface=public, service_id=3f5d2a62b9fa42d7a79ed0fe6157f400, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:02.886964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:02.888327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:02.900817 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:02.901882 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:02.902740 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 383/774] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:02 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:02.905588 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 392/775] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:02 2022] GET /identity/v3/endpoints?service_id=3f5d2a62b9fa42d7a79ed0fe6157f400&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:02.909816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:02.912038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:03.021486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd775-9e66-4e82-9ac9-ea13fc774e85 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:03.050170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:03.069005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:03.069625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.069899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.070166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.070801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] RBAC: Adding query filter params (name=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:03.071157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:03.072562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:03.082931 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 384/776] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:03.087274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:03.117571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d553fe-33e4-4544-8d25-b500991c0bc6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:03.182018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:03.182646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.182920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.183218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.184195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:03.186646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:03.192549 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 393/777] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:02 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:03.260610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463417b-73df-4b34-9897-3a6413ceff44 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:03.321798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:03.322336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.322606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.322867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.323691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] RBAC: Adding query filter params (type=network) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:03.323988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:03.325372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be42504-0448-4839-be2b-82f8ecc7db10 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:03.335601 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 385/778] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:03.341809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:03.420124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c87819-a37d-47e5-8e55-42b5706e9f42 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.420474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c87819-a37d-47e5-8e55-42b5706e9f42 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.420665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c87819-a37d-47e5-8e55-42b5706e9f42 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.422716 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 386/779] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:03.429174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.429454 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.429727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.446170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:03.452851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:03.454697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:03.497092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fed3a0-0213-4637-8495-c597daf0a595 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:03.524149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:03.532956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:03.533627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.533894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.534154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.534892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] RBAC: Adding query filter params (interface=public, service_id=29d97997de254ec2ab6b992bd6b2eb19, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:03.535136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:03.536634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:03.542188 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 394/780] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:03 2022] GET /identity/v3/endpoints?service_id=29d97997de254ec2ab6b992bd6b2eb19&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:03.547777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:03.551069 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 387/781] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:03.675288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82598cb6-3ac5-4473-9585-8c59e236493d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:03.698423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:03.721192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27c4e2c-3559-49ef-936d-b0a2969f9b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:03.721794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c4e2c-3559-49ef-936d-b0a2969f9b5a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:03.722064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c4e2c-3559-49ef-936d-b0a2969f9b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:03.722329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27c4e2c-3559-49ef-936d-b0a2969f9b5a None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:03.722930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c4e2c-3559-49ef-936d-b0a2969f9b5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:03.724398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27c4e2c-3559-49ef-936d-b0a2969f9b5a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:03.735858 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 395/782] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:03 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:03.860758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:03.999819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab63eec-b1fa-4c9c-a7f8-d4b8cfa9f532 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:04.057663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:04.059589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.060067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.060548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.072152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:04.078115 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:04.079086 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:04.079978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:04.080676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b036a-d2d3-4219-b7a0-90fafc25fdee None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.080676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b036a-d2d3-4219-b7a0-90fafc25fdee None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.080676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b036a-d2d3-4219-b7a0-90fafc25fdee None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.081501 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 396/783] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:04.087851 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 388/784] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:04.088206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.088693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.089045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.089847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:04.112939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:04.114428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:04.163542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:04.249591 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 397/785] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:04.267152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19dc7d7-f223-47b5-aaaa-979f1a2a76bb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:04.317113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:04.318138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.318617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.319099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.319995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:04.320492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:04.322143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:04.334983 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 389/786] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:04.342903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:04.472530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad49a15-b2ea-4966-9845-b44f63e4c50b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:04.501145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:04.524042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:04.525036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.525514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.525991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.537765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:04.541696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:04.544378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:04.567372 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:04.568399 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:04.569314 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:04.569314 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:04.569314 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:04.578062 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 398/787] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:04.593017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:04.697887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e91b0b-3ced-4525-9de9-f8ff11a4b0a0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:04.709104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:04.758648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:04.759738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.760234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.762280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.786695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:04.798063 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:04.799272 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:04.800175 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:04.805298 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 390/788] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:04.810169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c9bfc-973f-4c66-8bed-cb14bcd220a4 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:04.817077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:04.869994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:04.871131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.871672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.872145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:04.873013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:04.873540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:04.875293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:04.892509 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 399/789] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:04.903412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:04.958383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3249-472c-4ae2-ba09-20a5ff583f8d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:04.975349 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:04.998339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:04.998943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:04.999251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:04.999519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.000184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:05.000483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.001839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149fb62-5f4e-4877-b9a3-99d489c77762 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.010609 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 391/790] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:05.011859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876316-6c14-4e63-8c4b-a9f49a48bf98 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.012136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876316-6c14-4e63-8c4b-a9f49a48bf98 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.012406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94876316-6c14-4e63-8c4b-a9f49a48bf98 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.017608 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 392/791] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:05.018607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876316-6c14-4e63-8c4b-a9f49a48bf98 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.049529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:05.078685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4ff69-3250-4b02-bebf-bab0ff747cc5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.128937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94876316-6c14-4e63-8c4b-a9f49a48bf98 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.134913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.135792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.136203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.136651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.166444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.167977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.168961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.169428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.169940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3458361b-5895-4ac9-841f-3c69f5ddf574 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:05.180970 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:05.181971 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:05.182943 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 400/792] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:05.184454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.184565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.184792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.185934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.187718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.203624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:05.204790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:05.205877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:05.206809 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:05.208107 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:05.208107 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:05.208107 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:05.209898 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 393/793] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:05.210880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.287207 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 401/794] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:05.287207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.298751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:05.323742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:05.357253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a420ea-99fa-457f-8f5a-226e88c0f2e6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.420892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7130dad-a867-499c-879e-b5ff51ca0caa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.425844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.426721 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.427246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.427709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.428659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:05.429165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.430787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.469618 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 394/795] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:05.471656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.472622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.473102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.473602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.474444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:05.475587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.476892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.480121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.505190 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 402/796] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:05.510528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.622917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:05.650125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fb62f-ee73-4e7d-adae-dab9b31340b3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.665398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03447b2c-c421-43b8-876e-47c8d5cb2366 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.685662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.686608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.687074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.687582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.695589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.697284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.702352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.703043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.703346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.703615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:05.710164 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:05.711106 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:05.712109 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:05.714411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.714864 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 403/797] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:05.719549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.720173 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.727545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:05.733103 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:05.734190 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:05.735122 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:05.735122 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:05.735122 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:05.735122 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 395/798] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:05.740045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.833852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56203884-ba90-45e8-bb07-2417b6505f31 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.853923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bd7f1-25a7-426b-98c7-a61b74f4607e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:05.869663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.870742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.871251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.871687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.873819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:05.885156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:05.891294 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:05.892257 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:43:05.893283 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:05.893925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:05.893925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:05.893925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:05.893925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:05.894199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:05.894199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:05.895485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:05.896260 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 404/799] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:05.897232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.918817 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 396/800] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:05.919684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:05.973948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.001699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193bd256-a972-47ea-a8b1-918529a94770 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.028626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78605c34-359b-4c0e-9062-7a568e5ec15f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.036419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.037017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.037299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.037564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.038200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:06.038488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.039876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7023-2265-4e4b-aaee-607512dec052 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.056517 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 405/801] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:06.057803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6cfba-ae36-4bfd-9cba-585349be7f56 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.058091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6cfba-ae36-4bfd-9cba-585349be7f56 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.058372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6cfba-ae36-4bfd-9cba-585349be7f56 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.060108 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 406/802] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:06.060791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6cfba-ae36-4bfd-9cba-585349be7f56 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.069094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.070204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.070481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.070746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.071778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:06.072101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.073787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.087344 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 397/803] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:06.087344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.126144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.181600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6cfba-ae36-4bfd-9cba-585349be7f56 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.197114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d319b1a8-cf58-4809-af86-ad28c5329b47 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.211476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.221540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae6517-3390-4896-8a29-9ad8e055a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.222460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae6517-3390-4896-8a29-9ad8e055a854 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.222958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae6517-3390-4896-8a29-9ad8e055a854 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.223477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae6517-3390-4896-8a29-9ad8e055a854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.233566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.234200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.234472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.234735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.241584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.243024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ea0bf36a-451f-47f8-82c9-2bc4f27dddf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:06.254162 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:06.255066 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:06.256606 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 398/804] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:06.257207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.257207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.257375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.274492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:06.275716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:06.295603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae6517-3390-4896-8a29-9ad8e055a854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.297033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aae6517-3390-4896-8a29-9ad8e055a854 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.344308 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 399/805] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:06.345198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.459629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d91fce-d553-4177-b4c4-e6cf16ac95bf None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.461202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.477216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.496868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.497494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.497766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.498029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.508570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.510008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:06.524758 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:06.530818 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:06.531823 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:06.531823 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:06.531823 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:06.531996 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 407/806] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:06.531996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2065c1-47e6-4af0-ad84-48f520f81b59 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.531996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2065c1-47e6-4af0-ad84-48f520f81b59 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.531996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2065c1-47e6-4af0-ad84-48f520f81b59 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.531996 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 408/807] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:06.531996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2065c1-47e6-4af0-ad84-48f520f81b59 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.532362 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 400/808] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:06.532362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.636124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40c6c-04d7-4b48-9ce0-708ba9c624ac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.636794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2065c1-47e6-4af0-ad84-48f520f81b59 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.668999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.669613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.669834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.669960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.670108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.670421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.670695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.670794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:06.670966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.671100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.671639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:06.671940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.672535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.673289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba718c0-6e05-43b7-aab8-37eaa9c41eb9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.684820 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 409/809] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:06.685396 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 401/810] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:06.685532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.686514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.686788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.687050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.700501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.709244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:06.710592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:06.719781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.774549 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 402/811] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:06.775404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.788188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7fef3-e596-4830-ad7a-4cbc09b0664b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.825611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.826503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.826942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.827233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.837202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.838646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:06.852660 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:06.853629 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:06.854554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:06.854554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:06.854554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:06.858549 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 410/812] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:06.859393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.881010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45a1c-21da-4b81-a2d6-3682703f5584 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:06.915764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:06.916413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:06.916694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:06.916958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:06.917547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:06.918945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:06.927128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:06.927638 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:06.928655 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:06.929503 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 403/813] 10.222.0.23 () {56 vars in 1129 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:06.930315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:06.942290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:06.969692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8054d6f0-b4e5-4734-a1a3-07c450207602 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.005649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.006310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.006585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.006852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/512080b44c7542439333bd8b5f941496/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.008805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=512080b44c7542439333bd8b5f941496, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.010274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.025582 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 411/814] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:06 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/512080b44c7542439333bd8b5f941496/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:43:07.026133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.045428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553bd22-acd7-4615-ae3e-3d326ef7bf16 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.088098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.088745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.089018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.089286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.099272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.101638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:07.116700 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:07.117784 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 404/815] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:43:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:07.119479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.140108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0000c4a-4dfb-4c6d-96d9-ac6dc9a32a2b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.170193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.182709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.184168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.184593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.184908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.185573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:07.185875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.187308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.200650 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 412/816] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:07.201574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.211300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.248589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675bccc1-730b-45d3-9e70-3215cffb33d6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.284824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.285746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.286018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.286286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.286917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:07.287239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.288608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.293679 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 405/817] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:07.296019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.310306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ef661-e65f-4227-9e6b-49676b88075b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.345965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.346537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.346808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.347072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] PATH_INFO: `/v3/services/network` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.347728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.349031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:07.357957 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:07.358989 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:07.359896 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 413/818] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:07.360707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.408838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a4a13-a527-4a8d-a469-1bb0119d05cc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.413807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.438447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.445003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.445622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.446206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.446479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.447394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:07.447699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.449689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.472127 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 406/819] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:07.472346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102c468-dff6-42a3-b676-59b87b5ccf91 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.472444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.505913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.506456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.506723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.506986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.516034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.517477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:07.528783 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:07.529731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 414/820] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:07.530654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.595702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564cbee-5b2f-4994-b318-2279f2e7e5d1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.632603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.633197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.633468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.633731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.634350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:07.634642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.636375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.641318 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 407/821] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:07.642097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.674135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed364b-d97c-4263-9f16-cc9638ba17a2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.685182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.702468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.712803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.713413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.713681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.713943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.714576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] RBAC: Adding query filter params (name=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:07.714870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.716268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.720934 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 415/822] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:07.721527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.756300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc7b4b-50c0-4d7d-9873-662b8e35a186 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.789136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.789769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.790044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.790318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.802001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.803474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:07.819722 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:07.820757 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:07.821653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:07.821653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:07.821653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:07.821653 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 408/823] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:07.821878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.831850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ae925-4b18-4c4d-86f4-819f90a7da05 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.868751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.869467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.869802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.870221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.871062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:07.871439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.872935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:07.885693 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 416/824] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:07.886523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:07.931467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.949001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2b71a-f845-4a30-a74d-e03682edb878 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:07.957438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:07.988720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:07.989693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:07.989693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:07.989898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:07.990562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:07.991961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.016088 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 409/825] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:08.016983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:08.029446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f120-0d7b-4daf-af62-45263c75578a None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.078061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.078861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.079295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.079676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.080581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] RBAC: Adding query filter params (type=network) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:08.080970 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.082696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.093894 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 417/826] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:08.095149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:08.142043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f70339-faa8-4368-8fee-c9c1ccb631cc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.178194 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.178825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.179098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.179441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.180120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:08.180456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.181829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.200633 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 410/827] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:08.201672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:08.223819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:08.228444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:08.229835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106feed0-b326-447d-8efe-17aa92d94c09 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.270691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.271272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.271542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.271807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.272557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=c150143af9144547ab3dc5a5d318d45f) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:08.272854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.275170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.285137 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 418/828] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:08 2022] GET /identity/v3/role_assignments?user.id=c150143af9144547ab3dc5a5d318d45f&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:08.286032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:08.336395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc4a80-0912-4f86-bfdb-a61ad4318f60 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.390901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.392814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.392814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.397915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.398706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.400969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.437459 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 411/829] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:08 2022] POST /identity/v3/endpoints => generated 337 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:08.446578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052a761-f855-4437-b220-230a7dff77b5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.503215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.503900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.504168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.504427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.513798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.515714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:08.528798 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:08.529817 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:08.530729 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 419/830] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:08.536077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:08.636273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:08.664538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfda49-1d6f-44c9-9054-78e2b781363a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.703099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.704180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.704707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.705247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.706149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:08.706658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.708430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6644999b-bd76-4281-ab92-6d4f0d3dcb40 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.722944 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 412/831] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:08.729164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:08.748070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:08.837377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11814b9d-a549-424e-94d2-3a0663161f16 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:08.873940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d338ecd-0e34-410f-bb84-e5a1cc986cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:08.874607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338ecd-0e34-410f-bb84-e5a1cc986cea None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:08.874879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338ecd-0e34-410f-bb84-e5a1cc986cea None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:08.875147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d338ecd-0e34-410f-bb84-e5a1cc986cea None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/8f67555ada824965b62e7026cfbab0bf/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:08.890273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338ecd-0e34-410f-bb84-e5a1cc986cea None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=8f67555ada824965b62e7026cfbab0bf, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:08.891792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d338ecd-0e34-410f-bb84-e5a1cc986cea None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:08.910802 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 420/832] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:08 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/8f67555ada824965b62e7026cfbab0bf/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:43:09.235011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b5be-3e8a-46a9-af63-dfac9a053407 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:09.235610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b5be-3e8a-46a9-af63-dfac9a053407 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:09.235610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0b5be-3e8a-46a9-af63-dfac9a053407 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:09.237242 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 413/833] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:09.242513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:09.242798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:09.243069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:09.261410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:09.262618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:09.322914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:09.331995 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 421/834] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:09.605204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b5be-3e8a-46a9-af63-dfac9a053407 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:09.710752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0b5be-3e8a-46a9-af63-dfac9a053407 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:09.745281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:09.745911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:09.746184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:09.746452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:09.753421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:09.757155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:09.762172 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:09.763327 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:09.764383 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:09.766811 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 414/835] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:09.773139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:09.877848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec22ded-b502-46a5-830f-744eb95930fe None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:09.914573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:09.915457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:09.915865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:09.916269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:09.917303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:09.917750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:09.919269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:09.931943 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 422/836] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:09.941460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:10.039292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77ed5-4f8e-442e-a523-5384061749c8 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:10.075400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:10.076036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.076309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.076577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.086852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:10.088380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:10.103168 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:10.104158 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:10.105050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:10.105050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:10.105050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:10.105050 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 415/837] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:10.110068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:10.169752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:10.214660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07531ad-ab0c-439f-99a5-0a5069b4398c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:10.249957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:10.250573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.250844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.251106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.251782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:10.252134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:10.253514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:10.267042 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 423/838] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:10.276595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:10.329909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:10.381172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8442a9-68c6-4c6b-bbcb-882d3195ed45 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:10.415022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:10.415617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.415888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.416153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.424763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:10.426242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-402aa602-3c4f-4d58-a1d3-53f421b0781a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:10.439437 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:10.440503 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:10.441487 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 416/839] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:10.448716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:10.549613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced34f5-08c2-48a6-b3e5-6471db5a7c55 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:10.566493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86601a-778b-4d14-b978-aed2bd43062d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.566812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86601a-778b-4d14-b978-aed2bd43062d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.567054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86601a-778b-4d14-b978-aed2bd43062d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.568647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:10.569225 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 417/840] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:10.576182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e186f1-4237-42fb-8ae3-a2de9540009a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.576939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e186f1-4237-42fb-8ae3-a2de9540009a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.577418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e186f1-4237-42fb-8ae3-a2de9540009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.588880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:10.589505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.589777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.590032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.590775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:10.591073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:10.592548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:10.599528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-83e186f1-4237-42fb-8ae3-a2de9540009a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:10.600983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-83e186f1-4237-42fb-8ae3-a2de9540009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:10.611572 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 424/841] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Sep 11 00:43:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:10.617824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:10.684451 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 418/842] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:10.746704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9eb945-af3f-4b02-99ea-5d8b0918134c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:10.781849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:10.782500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.782741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.783023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.827251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:10.841845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0171a-5ab0-42f0-a6a5-6498a302a0dd None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.842162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0171a-5ab0-42f0-a6a5-6498a302a0dd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.842391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e0171a-5ab0-42f0-a6a5-6498a302a0dd None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.844019 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 419/843] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:10.849365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:10.849642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:10.849910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:10.853759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:10.855204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:10.872422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:10.874265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:10.920200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:10.960037 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 425/844] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:10.960917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:10.963130 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 420/845] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:10.968556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.059713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.080349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b516e-7132-4055-aedb-2bf1c2764b17 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.090501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a690f-71d0-460c-a2fd-cd36aa9ae286 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.119106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.119953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.120233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.120504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.131972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.133466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.135095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.135783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.136055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.136320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.136924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.138355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:11.150601 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:11.151749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.158366 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 426/846] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:11.159218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:11.167429 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:11.168499 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:11.169928 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:11.169928 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:11.169928 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:11.174268 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 421/847] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:11.174861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.202633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.283436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d08cb-4233-4029-a3fb-887e1de6bdbc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.286731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c6aa4-8b70-4801-88d3-b0a98f9dff80 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.317009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.317616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.317884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.318149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.318784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:11.319081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.319591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.320190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.320460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf724b-113e-4a6c-98b7-841d84e72521 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.320560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.320723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.321343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:11.321639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.322982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.327298 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 427/848] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:11.328063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.330718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.336413 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 422/849] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:11.337641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7680b-461b-4959-b5f7-0ccecadb5886 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.337919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7680b-461b-4959-b5f7-0ccecadb5886 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.338187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7680b-461b-4959-b5f7-0ccecadb5886 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.339398 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 423/850] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:11.340082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7680b-461b-4959-b5f7-0ccecadb5886 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.418679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.430832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338e021-eea9-4e5f-a4e9-2d8842afd328 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.438057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7680b-461b-4959-b5f7-0ccecadb5886 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.463592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.464312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.464591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.464858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.465440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.466836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.471274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.471922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.472487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.472487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.473113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:11.473343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.474682 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4437ec0-29b7-4ba7-9a27-f20708dac56f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:11.475724 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:11.476689 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.477687 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 428/851] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:11.478494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.495313 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 424/852] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:11.495313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.495313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.495313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.515127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:11.517532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:11.550920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.592579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12e3b8-e2b4-4021-94dd-1ce281167218 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.603401 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 425/853] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:11.603401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.634234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.635241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.635588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.636365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.647440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.649023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:11.664067 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:11.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:11.665945 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 429/854] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:11.671875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.691860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.720524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34150c96-982d-44d8-8b5f-6e67db49a9de None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.765066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.765769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.766108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.766435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.767718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:11.768086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.770345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.777118 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 426/855] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:11.782588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.796769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110b9d6-f4ee-4aac-9e08-61e19949d6ab None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.824391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.836256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.836713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.837156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.838024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:11.838506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.840157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.858379 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 430/856] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:11.859606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.893025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76bdd0-821a-4cf1-8276-9fa731643843 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:11.931462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:11.932065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:11.932334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:11.932732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:11.933352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:11.934002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:11.935412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:11.941041 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 427/857] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:11.949212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:11.952408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:11.979165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ae4c0-c605-4907-a46c-8e5dd920bb3f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.029031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.029737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.030070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.030343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.031086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=512080b44c7542439333bd8b5f941496) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:12.031419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:12.032911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:12.042478 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 431/858] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/role_assignments?user.id=512080b44c7542439333bd8b5f941496&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:12.066104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a42d9-9cf5-4921-8003-0e806a734504 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.108141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.108710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.108984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.109249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.122962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:12.128829 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:12.129870 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:12.131144 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 428/859] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:43:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:12.136609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:12.200885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:12.255393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f58ad5-2c50-49db-8dd2-b5fbae423cba None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.290849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.291496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.291798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.292080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.292799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:12.293100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:12.294471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:12.303598 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 432/860] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:43:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:12.309894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:12.374817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:12.415725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade62925-8986-491c-b9d2-784a08a9d9da None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.450847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.451509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.451781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.452044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.463092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:12.464570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:12.479220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:12.480216 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:12.481558 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:12.481558 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:12.481558 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:12.481558 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 429/861] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:12.486453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:12.594591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35af4bd-087c-407f-a210-28086d18f584 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.608153 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:12.633398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.634735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.635510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.636104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.636964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:12.637509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:12.639080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:12.651844 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 433/862] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:12.659825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:12.758850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca82c7-6363-4a73-92eb-30dd6dca9308 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.775997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:12.793913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.794543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.794817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.795080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.803526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:12.805126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:12.817199 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:12.818177 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:12.819769 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 430/863] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:43:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:12.824357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:12.924326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718421b-edf3-405f-9fa8-fa7de11389fa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:12.961539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:12.962495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:12.962943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:12.963407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:12.964364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:12.964850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:12.966442 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:12.984817 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 434/864] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Sep 11 00:43:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:12.990663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:13.094473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd15a7-e8c6-42cd-a8e2-26f614e5856a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:13.128618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:13.129497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.129943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.130403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.183038 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:13.208637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:13.210468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:13.314634 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 431/865] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:13.320067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:13.352304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:13.426559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8b4e0-5cf5-4c2c-9a4d-10ba21159f05 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:13.462836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:13.463728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.464167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.464582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.475486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:13.477106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:13.492887 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:13.494098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:13.495281 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:13.495281 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:13.495281 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:13.496960 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 435/866] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:13.507220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:13.577035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:13.600328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b22a63-4260-403a-9538-54cb3d0ded38 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:13.635568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:13.636194 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.636462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.636727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.637386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:13.637675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:13.639059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383462b6-e8b4-4cdc-8c83-89c52a658c01 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:13.652382 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 432/867] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:13.657695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:13.752729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:13.772499 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e02e1-7962-4bef-9981-c583e6b7fe8c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:13.778130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d07bd-9c56-4d30-bb68-42e1d7416d31 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.778407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d07bd-9c56-4d30-bb68-42e1d7416d31 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.778680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d07bd-9c56-4d30-bb68-42e1d7416d31 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.784453 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 433/868] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:13.785933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.786214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.786485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.809423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:13.810634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:13.812504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:13.813385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.813854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.814292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.821729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:13.823439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:13.835085 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:13.836525 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:13.837436 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:13.844435 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 436/869] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:13.845537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:13.890812 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 434/870] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:13.953669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a6d09-a9d6-4feb-bcc5-ddcbc9777f29 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:13.988179 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:13.988790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:13.989066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:13.989334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:13.990027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:13.990327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:13.991957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:14.004510 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 437/871] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:14.010465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:14.074951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:14.109573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6dc27-4418-40f3-bd78-d4c885e584f1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:14.147436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:14.148047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.148383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.148652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.149388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=8f67555ada824965b62e7026cfbab0bf) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:14.149689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:14.151110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d531912-fc92-4a99-8394-d8a232614cae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:14.157736 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 435/872] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:14 2022] GET /identity/v3/role_assignments?user.id=8f67555ada824965b62e7026cfbab0bf&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:14.202962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:14.207047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:14.325164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a93da-f598-40c1-aec5-3084d9a825d8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.325794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a93da-f598-40c1-aec5-3084d9a825d8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.325794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a93da-f598-40c1-aec5-3084d9a825d8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.327370 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 436/873] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:14.331034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69771515-e2ca-4e89-a0e0-bcfd8481dcb1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:14.333055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.333336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.333608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.352622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:14.353854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:14.377269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549d1a7-4bb8-47bf-af9a-40f3927709dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:14.377924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d1a7-4bb8-47bf-af9a-40f3927709dd None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.378199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d1a7-4bb8-47bf-af9a-40f3927709dd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.378468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d1a7-4bb8-47bf-af9a-40f3927709dd None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.380241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d1a7-4bb8-47bf-af9a-40f3927709dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:14.381347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d1a7-4bb8-47bf-af9a-40f3927709dd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:14.398981 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 438/874] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:43:14 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:14.400267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deea377-e57a-4ac2-8d81-9a547f2e2d27 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.400546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deea377-e57a-4ac2-8d81-9a547f2e2d27 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.400816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deea377-e57a-4ac2-8d81-9a547f2e2d27 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.408215 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 439/875] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:14.414426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.414776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.415420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.442426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:14.443761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:14.450337 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 437/876] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:14.518120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:14.533805 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 440/877] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:14 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:14.719401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:14.777544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:14.805524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:14.824898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5dc746-0e6b-462d-84fa-ec84c2dede8b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:14.859903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:14.860911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.861466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.861931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.862711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:14.864346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:14.873669 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:14.874681 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:14.875543 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:14.878338 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 438/878] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:14.884368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:14.913723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bbabf-b815-43d4-a717-79b0f538ac60 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:14.952043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:14.953100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:14.953543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:14.953982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:14.955201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:14.956615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:14.983602 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 441/879] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:43:14 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:14.992028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fc16a-c243-4519-9935-9d79206562b7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:15.019897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:15.037109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:15.038144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:15.038619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:15.039109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:15.040010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:15.040514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:15.042132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ca9a3-847e-45d3-b08f-13a1cc850648 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:15.048342 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 439/880] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:15.055438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:15.121415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:15.178205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0f5a2-b2a1-4e01-832a-56538fa3e357 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:15.214098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:15.214717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:15.214991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:15.215286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:15.215925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:15.216224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:15.217655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e01fbf-ad19-406e-a021-d4a4c1fba18d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:15.223209 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 442/881] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:16.593622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b6cd-7a6a-4258-a6e2-7a7327652cf8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:16.594218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b6cd-7a6a-4258-a6e2-7a7327652cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:16.594218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059b6cd-7a6a-4258-a6e2-7a7327652cf8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:16.595813 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 440/882] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:16.597759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2b9b1-fd18-4f42-8086-7eb4db2a81bc None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:16.598176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2b9b1-fd18-4f42-8086-7eb4db2a81bc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:16.598346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2b9b1-fd18-4f42-8086-7eb4db2a81bc None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:16.599897 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 443/883] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:16.602760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:16.603332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:16.603900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:16.604890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:16.605172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:16.605446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:16.623594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:16.624041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:16.624788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:16.625617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:16.711203 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 441/884] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:16.715823 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 444/885] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:16.972322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:16.981076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.036706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.085566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95612533-3dbb-46a5-85e1-f2a8ecd84e7d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.097859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.103916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a1b24-dd97-41ac-93ba-fa742142a471 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.130510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.131607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.132097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.132583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.146955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:17.149440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.150087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.150362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.150636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.151262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.152643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:17.153676 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:17.154691 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:17.155653 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:17.163725 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 442/886] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Sep 11 00:43:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:17.164879 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:17.165974 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:17.167570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:17.168896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.170872 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 445/887] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:17.171802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.288396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b09de-b7f8-4955-8fe5-47e23c127c7c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.292102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b84aa-695e-481e-949a-1bb014e455e3 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.304845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.328940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.329969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.330549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.331028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.331875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.332175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:17.332477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.332744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.333006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.333305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.333632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:17.333925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.335300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d5d8a-55f8-4b86-9c17-ef5e52959027 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.335531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.340453 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 446/888] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:17.342106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd31d40-8857-4cf9-bcde-49641e792d21 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.342377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd31d40-8857-4cf9-bcde-49641e792d21 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.342639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd31d40-8857-4cf9-bcde-49641e792d21 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.344395 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 447/889] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:17.345193 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 443/890] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:17.350552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.350894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.351003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.351246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.370306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:17.371504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:17.386181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.438616 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 448/891] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:17.439628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.455552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3936aaa-38e4-4f25-a5f1-787102757ae1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.489671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.490370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.490650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.490924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.491585 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:17.491890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.493266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c118d2-e6d5-482c-a3a4-32def59b7c0b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.498385 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 444/892] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:17.499398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76d5d-fe02-484c-b71b-20e12b87672d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.499653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76d5d-fe02-484c-b71b-20e12b87672d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.499921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c76d5d-fe02-484c-b71b-20e12b87672d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.501314 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 445/893] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:17.503875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76d5d-fe02-484c-b71b-20e12b87672d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.559913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48764403-c7ba-41ac-b2ad-6a58d94eaf67 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.562413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.597612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.598237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.598799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.598799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.605973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.610958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.612679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.613443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c76d5d-fe02-484c-b71b-20e12b87672d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7eded0ec-f9dc-4c4c-83c5-143cd099efdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:17.630077 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:17.631050 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 449/894] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.637225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.648372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.648977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.649249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.649518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.650226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] RBAC: Adding query filter params (interface=public, service_id=3d81e589914a41cb9255b8154f975398, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:17.650525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.651935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.653166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:17.654334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:17.664085 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 446/895] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/endpoints?service_id=3d81e589914a41cb9255b8154f975398&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:17.664085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.723406 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 450/896] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:17 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:17.724249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.768615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0528b-8b74-4156-8906-ede252b3a998 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.802544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.803150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.803452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.803715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.804349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:17.804643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.806064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.821119 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 447/897] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:17.821939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.825540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.843453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731cd9e-ac3e-4a7c-a843-1bc6e9e499e3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.846862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:17.877594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.878202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.878485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.878730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.879363 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.880719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:17.886499 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 451/898] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:43:17.887491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:17.954105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e037dab-0ba7-4dc5-b0a2-ddb20bb79873 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:17.995826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:17.996476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:17.996746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:17.997238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:17.997848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:17.999767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.009688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afefd9-385a-4f3c-a3d0-6d963e257249 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:18.011317 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:18.012493 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:18.013794 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:18.017459 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 448/899] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:18.018446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.052693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.053328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.053613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.053859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.062531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.064025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:18.076714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:18.077929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 452/900] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:43:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:18.079098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.093896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.118303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.140287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c6865-ffcd-459a-b73a-a614a6dd53c7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.182751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.183838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.184042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.184311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.184972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.186341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.189967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f4861-a893-498f-b5f1-fc7a21cb4617 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.203212 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 449/901] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:43:18 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:18.204611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.247831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.248474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.248780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.249042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.249731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:18.250033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.251454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.261214 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 453/902] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:18.267344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.333260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfc050-ae6a-4467-b695-dcc660d4d455 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.373472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.379416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.380286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.380732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.381138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.382022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:18.382462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.384979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.405625 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 450/903] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:18.411270 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.414241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.419046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23b6b9-c091-472f-80d5-4a37e8c55117 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.457762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.458442 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.458714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.458977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.459649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:18.459947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.461313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.467346 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 454/904] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:18.472891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.522279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a9d00e-6c17-42c5-bafe-f42d9d50819e None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.561420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.562404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.563002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.563488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.603854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15e418-4710-49b0-9a20-cc577082df0f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.638609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.640192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.640602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.641440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.642033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.642387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.643698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.645550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] RBAC: Adding query filter params (interface=public, service_id=51a094d8cc164c058104fe364f41fa3f, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:18.646090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.647756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.649695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.665941 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 455/905] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/endpoints?service_id=51a094d8cc164c058104fe364f41fa3f&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:18.667102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.757999 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 451/906] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 00:43:18.763263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:18.809523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961e1fd9-5443-4521-8870-d71e9dd963ec None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.855539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.856178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.856551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.856886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.857727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.859742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.871225 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 456/907] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/services => generated 1497 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:43:18.894601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67386ea5-4458-41d3-b912-2aa17655d7e4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:18.896627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:18.932292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:18.932946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:18.933215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:18.933816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:18.945463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:18.946989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:18.963229 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:18.964288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:18.965251 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:18.965251 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:18.965251 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:18.965251 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 452/908] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:18.972078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:19.042239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:19.086579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3c2e5-a934-4a4c-a542-537342beaeda None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:19.123118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:19.124095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:19.124585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:19.125067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:19.125928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:19.126444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:19.128129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:19.142458 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 457/909] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:19.148388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:19.255301 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854a8b9-de0b-429f-9c00-49d8ef0ca354 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:19.289235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:19.290146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:19.290586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:19.291010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:19.298023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:19.299708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:19.311806 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:19.312943 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:19.314183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:19.316268 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 453/910] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:19.322250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:19.429058 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc232c-d9cc-4a3f-bea7-7ccda4339122 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:19.453177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:19.466126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:19.466964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:19.467431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:19.467863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:19.468708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:19.469174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:19.470767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ec384-5a9a-4ed1-8430-aa1f0c6ecb15 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:19.487570 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 458/911] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:19.489796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:19.506581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:19.593199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722002e6-35b5-48df-b518-cd2b2c3000ae None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:19.627539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:19.628397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:19.628845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:19.629245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:19.630143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] RBAC: Adding query filter params (role.id=4ddea9e8dc7642d0bca59749640626d7, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=8f67555ada824965b62e7026cfbab0bf) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:19.630591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:19.632180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e8495-da8d-42b5-a77a-175ed84b9873 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:19.638589 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 454/912] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:19 2022] GET /identity/v3/role_assignments?user.id=8f67555ada824965b62e7026cfbab0bf&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=4ddea9e8dc7642d0bca59749640626d7 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:20.233339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa6300-d4b8-4175-87e3-ee4183e56a8c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:20.233902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa6300-d4b8-4175-87e3-ee4183e56a8c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:20.233902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa6300-d4b8-4175-87e3-ee4183e56a8c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:20.235394 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 459/913] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:20.240876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78532fc-87b4-46cd-bfda-d642aa510dac None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:20.241650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78532fc-87b4-46cd-bfda-d642aa510dac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:20.242132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78532fc-87b4-46cd-bfda-d642aa510dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:20.261048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d78532fc-87b4-46cd-bfda-d642aa510dac None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:20.262619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d78532fc-87b4-46cd-bfda-d642aa510dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:20.335280 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 455/914] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:20.549126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcce7b-dcfa-45f5-a544-8c3bf668bd3a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:20.549856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcce7b-dcfa-45f5-a544-8c3bf668bd3a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:20.550338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcce7b-dcfa-45f5-a544-8c3bf668bd3a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:20.552179 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 460/915] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:20.561824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:20.562386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:20.562876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:20.582313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:20.583846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:20.585984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:20.637042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcce7b-dcfa-45f5-a544-8c3bf668bd3a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:20.659225 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 456/916] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:20.748504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcce7b-dcfa-45f5-a544-8c3bf668bd3a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:20.783054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:20.783977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:20.784394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:20.784830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:20.785565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:20.787083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:20.796235 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:20.797458 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 11 00:43:20.798520 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:20.801005 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 461/917] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:20.808372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:20.915949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7afa98-f4f1-4415-8a9f-290c40904a79 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:20.929183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:20.952469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:20.953084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:20.953359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:20.953629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:20.954262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:20.954569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:20.956086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:20.965957 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 457/918] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:20.967215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:20.969885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:20.982214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:21.045380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce9dc2-432e-4e88-ba0f-dc8b247fb02d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.082379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6274c6-c700-4500-b4c1-6a2c24e151e2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.097013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.097952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.098433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.098910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.099728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.101299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:21.110178 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:21.111262 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:21.112133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:21.119166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.119339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.119559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.119767 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 462/919] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:21.119864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.120604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:21.120911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.121797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.122274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.131263 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 458/920] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Sep 11 00:43:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:21.134112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.214386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:21.252769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0063f1-3c5d-4fe8-ac91-f2e3f9c4da5c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.258419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5360c-9f5d-426f-b63c-6d02587ff8f9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.270660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:21.289358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36265733-6a9c-40ed-9422-79904372aced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.289970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36265733-6a9c-40ed-9422-79904372aced None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.290399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36265733-6a9c-40ed-9422-79904372aced None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.290676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36265733-6a9c-40ed-9422-79904372aced None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.291680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36265733-6a9c-40ed-9422-79904372aced None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.293045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36265733-6a9c-40ed-9422-79904372aced None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.295784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.296559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.296829 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.297092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.297765 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:21.298067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.299542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd789ab-435e-40b0-94f1-a059bd2e6b9e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.305014 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 463/921] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:21.306399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d5d7-5f5c-4512-a5e1-49ab4752fb9a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.306678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d5d7-5f5c-4512-a5e1-49ab4752fb9a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.306928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825d5d7-5f5c-4512-a5e1-49ab4752fb9a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.308835 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 464/922] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:21.313467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d5d7-5f5c-4512-a5e1-49ab4752fb9a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.318155 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 459/923] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:21.318709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.318986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.319294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.348330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:21.349480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:21.432344 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 460/924] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:21.452289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825d5d7-5f5c-4512-a5e1-49ab4752fb9a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.493108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.494170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.494663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.495130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.496061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:21.496626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.498232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.504884 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 465/925] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:21.510535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.565529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:21.621302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ce7bb-a149-475c-ab55-23619075dde2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.656189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.656807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.657083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.657346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.658078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] RBAC: Adding query filter params (interface=public, service_id=ead204df1f11413987ef08994edb6c20, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:21.658385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.659796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.665607 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 461/926] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:21 2022] GET /identity/v3/endpoints?service_id=ead204df1f11413987ef08994edb6c20&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:21.671358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.705715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.717013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:21.788933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b49b8-c314-4b9f-b132-37dc1605b42c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.823923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f031d1-f89e-4b4b-a6e9-d5097c6dac48 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:21.826133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62613b93-95d9-42fc-8f65-6798a4b7fbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.826925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62613b93-95d9-42fc-8f65-6798a4b7fbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.827297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62613b93-95d9-42fc-8f65-6798a4b7fbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.827637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62613b93-95d9-42fc-8f65-6798a4b7fbb2 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.828317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62613b93-95d9-42fc-8f65-6798a4b7fbb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.829773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62613b93-95d9-42fc-8f65-6798a4b7fbb2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.836620 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 466/927] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:21 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:43:21.839783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7539f21-f820-49e8-b575-3a76e5ee3b5f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.840067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7539f21-f820-49e8-b575-3a76e5ee3b5f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.840344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7539f21-f820-49e8-b575-3a76e5ee3b5f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.842112 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 467/928] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:21.842419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:21.847772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.848074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.848440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.865188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:21.866229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:21.866721 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:21.867245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:21.868049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:21.869383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:21.870206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:21.870764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:21.884322 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:21.885333 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 11 00:43:21.886424 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:21.895380 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 462/929] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Sep 11 00:43:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:21.900512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:21.967233 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 468/930] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:22.021045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d615a2-3424-4f12-a2d4-8133d561118d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.056372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.056979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.057296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.057630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.058249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:22.058911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:22.060326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:22.066637 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 463/931] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:22.072310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.105267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:22.176431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f534da-c1f4-4c7e-ac40-646625efe47b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.212304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.212920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.213189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.213473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.214200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:22.214492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:22.215902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:22.221696 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 469/932] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:22.227626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.246017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.275533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:22.344027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89c299-f475-41ed-8689-81d1319561a6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.363044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:22.369781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aae3a8-9c7e-4194-8c3c-f62bf890d5a5 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.382960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.383608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.383885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.384155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.384843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:22.386212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:22.411449 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 464/933] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:22 2022] POST /identity/v3/endpoints => generated 347 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:22.416932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.417884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.418295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.418697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.431571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:22.437639 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:22.438857 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:22.439924 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:22.442439 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 470/934] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Sep 11 00:43:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:22.448799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.559210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:22.567464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8de2cb-c052-427a-a9b5-f02ce4864ab1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.614324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.615211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.615623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.616056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.616863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:22.617294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:22.618784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:22.628174 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 465/935] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:43:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:22.634016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.744455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26df93-60e6-4f4a-9a8b-57ee1dfd75d6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.780372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.781247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.781659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.782164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.793039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:22.794697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:22.810046 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:22.811254 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:22.812375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:22.812375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:22.812375 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:22.814060 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 471/936] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:22.820094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.926148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cdcd1e-779e-4e47-8f92-b72b4ec193ce None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:22.960172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:22.960781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:22.961052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:22.961319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:22.961960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:22.962256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:22.963666 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:22.976138 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 466/937] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:22.981827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:22.987466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:23.087670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c394fb-0f23-4022-8b7d-35645ffa2732 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:23.124713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:23.125350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.125623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.125890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.133825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:23.135301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:23.147332 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:23.148315 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:23.149259 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 472/938] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:43:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:23.156469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:23.178403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:23.268618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac4964-5d8b-4d9c-9cc8-f6be77ce23a7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:23.303773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:23.304415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.304706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.304980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.305758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:23.306063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:23.307516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:23.325880 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 467/939] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Sep 11 00:43:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:23.331531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:23.401750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:23.434267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead639a4-36cd-4de5-a3ec-96d6b319e0c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:23.471268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:23.471910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.472165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.472430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.483871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:23.485309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cf7f4f97-4f3f-43d0-bdbb-97ac513f6f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:23.499602 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:23.500607 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:23.502261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:23.502261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:23.502261 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:23.502261 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 473/940] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:23.506904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:23.591585 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:23.617280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6617f-b9b6-40cc-86a9-28b39cc3675d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:23.637707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ba84-2e9f-4e9d-ac81-111bcbcf9792 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.638033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ba84-2e9f-4e9d-ac81-111bcbcf9792 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.638254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af81ba84-2e9f-4e9d-ac81-111bcbcf9792 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.639980 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 474/941] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:23.644974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.645253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.645521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.655904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:23.656802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.657276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.657757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.658798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:23.659376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:23.660945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:23.663993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:23.665182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:23.677137 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 468/942] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:23.682738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:23.747435 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 475/943] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:23.788920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa184-3391-43cb-bc40-08adb551f94b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:23.824142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:23.824773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.825041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.825304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:23.828082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:23.832874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:23.834485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:23.846048 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:23.846987 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:23.847937 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:23.851496 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 469/944] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:23.853381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:23.902960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:23.961728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd8c76-9c57-4f6e-b944-33b7e11c7b7c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:23.997115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:23.998244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:23.998722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:23.999208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.000101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:24.000646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.002324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.014646 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 476/945] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:24.020378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:24.036484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:24.128691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc6c2cd-596f-4cfd-9607-2a581c3e7bdb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:24.163911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c64fd-e94f-4ac7-a08e-6048c8d1c60d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:24.166141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3965-9118-431d-b0cb-f9a8dfe03b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:24.167591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3965-9118-431d-b0cb-f9a8dfe03b09 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.167591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3965-9118-431d-b0cb-f9a8dfe03b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.167591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3965-9118-431d-b0cb-f9a8dfe03b09 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/8f67555ada824965b62e7026cfbab0bf/roles/4ddea9e8dc7642d0bca59749640626d7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.169629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed3965-9118-431d-b0cb-f9a8dfe03b09 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=8f67555ada824965b62e7026cfbab0bf, role_id=4ddea9e8dc7642d0bca59749640626d7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.171054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed3965-9118-431d-b0cb-f9a8dfe03b09 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.173127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:24.199571 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 470/946] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:24 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/8f67555ada824965b62e7026cfbab0bf/roles/4ddea9e8dc7642d0bca59749640626d7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:43:24.201792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af62867-b61b-4d8b-91b6-4b187b2c02bc None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.202340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af62867-b61b-4d8b-91b6-4b187b2c02bc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.202618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af62867-b61b-4d8b-91b6-4b187b2c02bc None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.204294 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 471/947] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:24.205335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:24.206164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.206584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.207006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.207842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.209503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.210581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.210872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.211143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.213011 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING keystone.server.flask.common [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 00:43:24.234712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:24.236144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:24.238193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.sql.core [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 00:43:24.238193 dv-cinder-master-3-smb devstack@keystone.service[81488]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 00:43:24.238193 dv-cinder-master-3-smb devstack@keystone.service[81488]: [parameters: {'id': 'b87b717d24944867bf12c7eefa5524a7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 00:43:24.238193 dv-cinder-master-3-smb devstack@keystone.service[81488]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81488) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b87b717d24944867bf12c7eefa5524a7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b87b717d24944867bf12c7eefa5524a7. Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 00:43:24.248555 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 11 00:43:24.249497 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application yield resource Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application next(self.gen) Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise self.value Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application session.commit() Sep 11 00:43:24.250488 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.session.flush() Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application compat.raise_( Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 00:43:24.251464 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 00:43:24.258623 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application conn.query(q) Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application result.read() Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 00:43:24.259621 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [parameters: {'id': 'b87b717d24944867bf12c7eefa5524a7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.260815 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:24.261819 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b87b717d24944867bf12c7eefa5524a7. Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.262946 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 477/948] 10.222.0.23 () {60 vars in 1140 bytes} [Sun Sep 11 00:43:24 2022] POST /identity/v3/projects => generated 262 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:24.265611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:24.316371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:24.343260 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 472/949] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:24.396252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531c801-af64-4b99-bc62-9d7ae35c1166 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:24.440704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:24.441374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.441597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.441849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.448910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.450553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:24.461946 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:24.462839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.463845 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 478/950] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:24.470752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:24.573005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3c846-06d5-44a1-8349-0381c565b0ec None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:24.608673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:24.609998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.610371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:24.610702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.611335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.612223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:24.612734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.615753 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37100c49-74d0-4c58-bf81-0e34bed625dd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.637608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:24.640582 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 473/951] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:24.734667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3780a3-d057-4784-b406-d4c6e9228c13 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:24.773643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:24.774303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c16b92-82ff-4878-a76f-9934896e54a1 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.774587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c16b92-82ff-4878-a76f-9934896e54a1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.774860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c16b92-82ff-4878-a76f-9934896e54a1 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.775035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.775480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.775908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.776502 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 474/952] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:24.776846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.779201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.781956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.782165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.782432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:24.790588 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:24.792097 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:24.793255 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:24.795696 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 479/953] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:24.803101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:24.805759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:24.807102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:24.863679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:24.906133 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 475/954] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:24.909440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:24.927833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed14213-7e4a-494f-a58c-9fafb6d3a804 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:24.964957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:24.965571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:24.965841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:24.966107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:24.966739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:24.967038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:24.968467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:24.976860 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 480/955] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:24.983920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:25.087426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bafe58-892a-4eee-813c-d140aba72c62 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:25.121409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:25.122030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:25.122316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:25.122588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:25.123246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:25.123554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:25.124931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:25.130566 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 476/956] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:25.136647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:25.175732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:25.223702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:25.271551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b006cc-7945-48db-a0a9-04cf164b01eb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:25.279246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b07cf0-e531-48f9-85d6-9da9ceef64cb None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:25.317788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:25.318440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:25.318670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:25.318933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:25.319592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:25.320047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:25.320815 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:25.321065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:25.321482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:25.321965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:25.322982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:25.325090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:25.330090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:25.331405 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:25.332552 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:25.336376 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 477/957] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:25.337223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:25.357779 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 481/958] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:25 2022] POST /identity/v3/endpoints => generated 349 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:25.457109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29148c76-bba3-48ae-89d6-6de819192bac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:25.459953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:25.503090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:25.503751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:25.504076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:25.504344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:25.504980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:25.505285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:25.506761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1973b-ebc9-43b3-823f-864fcaa28723 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:25.512970 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 478/959] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:25.518961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:25.529992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:25.633083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bfbb7-b8af-42e3-a46c-3e9705ae1ed1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:25.668851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:25.669838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:25.670341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:25.670806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:25.671679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:25.672208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:25.673819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbc19f-9676-4ef7-9b85-6108e792a3f6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:25.678982 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 482/960] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:26.439404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea514806-4ff2-4cd8-990c-ed2ff3444f87 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:26.440256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea514806-4ff2-4cd8-990c-ed2ff3444f87 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:26.440664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea514806-4ff2-4cd8-990c-ed2ff3444f87 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:26.442359 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 479/961] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:26.448061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7501d25-febb-46b0-815b-73670875896c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:26.448343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7501d25-febb-46b0-815b-73670875896c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:26.448621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7501d25-febb-46b0-815b-73670875896c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:26.467289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b7501d25-febb-46b0-815b-73670875896c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:26.468814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b7501d25-febb-46b0-815b-73670875896c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:26.533176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:26.539236 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 483/962] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:26.802725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea514806-4ff2-4cd8-990c-ed2ff3444f87 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:26.907725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea514806-4ff2-4cd8-990c-ed2ff3444f87 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:26.945484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5400a5-4f95-4029-a369-0770d19f4a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:26.946318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5400a5-4f95-4029-a369-0770d19f4a3c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:26.946590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5400a5-4f95-4029-a369-0770d19f4a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:26.946861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5400a5-4f95-4029-a369-0770d19f4a3c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:26.959542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5400a5-4f95-4029-a369-0770d19f4a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5c5400a5-4f95-4029-a369-0770d19f4a3c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:26.964449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:26.965671 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:26.966750 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 480/963] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Sep 11 00:43:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:26.973893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7501d25-febb-46b0-815b-73670875896c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.071935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7501d25-febb-46b0-815b-73670875896c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.080420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968bf2-c5fd-4f57-9e9a-f50cca9f1714 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.080986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968bf2-c5fd-4f57-9e9a-f50cca9f1714 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.081451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968bf2-c5fd-4f57-9e9a-f50cca9f1714 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.083855 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 481/964] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:27.089080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.089638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.090095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.107644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.108306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.108586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.108855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.109530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:27.109636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:27.109818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.111317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.111678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:27.120147 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 484/965] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:43:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:27.125652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.133813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:27.207612 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 482/966] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:27.210051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:27.252880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393556b-3bdf-4cb9-8bc3-8729baf3a381 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.287554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.288536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.289011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.289497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.301076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.302866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:27.318561 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:27.319647 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:27.320570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:27.320570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:27.320570 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:27.322855 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 485/967] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:27.328340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.427558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59b361-da29-47d9-9b56-72c6b8bf406b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.464995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.465650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.465908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.466148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.466804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:27.467097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.468550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.482168 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 483/968] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:27.487889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.505113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.526950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:27.605583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc533d-52c6-4d15-9207-33ba4416fa82 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.619260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:27.629529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c509be9-0f13-43d4-bda2-6cf568744447 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.640033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.640661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.640930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.641194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.648901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.650332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-01bec94a-cc87-48f0-bbdd-dcc5bec5e034 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:27.661499 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:27.662491 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 486/969] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:43:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f6258-c2aa-4905-9e25-078cc8a03787 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.664256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f6258-c2aa-4905-9e25-078cc8a03787 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.665041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f6258-c2aa-4905-9e25-078cc8a03787 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.665653 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 487/970] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:27.668968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.669586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.669847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.670106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.670290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f6258-c2aa-4905-9e25-078cc8a03787 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.670744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.672312 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.674938 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING keystone.server.flask.common [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 11 00:43:27.688626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.sql.core [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 00:43:27.688626 dv-cinder-master-3-smb devstack@keystone.service[81487]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 00:43:27.688626 dv-cinder-master-3-smb devstack@keystone.service[81487]: [parameters: {'user_id': '31372e51e0264ce99dc5d7ec868aac9b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 00:43:27.688626 dv-cinder-master-3-smb devstack@keystone.service[81487]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81487) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-01c046b4-2047-4410-8602-492dd84bc71e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Sep 11 00:43:27.705386 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 00:43:27.706370 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application yield resource Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application next(self.gen) Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.rollback() Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.force_reraise() Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise self.value Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application session.commit() Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 11 00:43:27.708056 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._prepare_impl() Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.session.flush() Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._flush(objects) Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application compat.raise_( Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application flush_context.execute() Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rec.execute(self) Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 11 00:43:27.709318 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 11 00:43:27.710231 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application result = self._query(query) Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application conn.query(q) Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application result.read() Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 11 00:43:27.711105 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [parameters: {'user_id': '31372e51e0264ce99dc5d7ec868aac9b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:27.714484 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 11 00:43:27.716264 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 484/971] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Sep 11 00:43:27 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.717489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.727994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:27.729566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:27.795595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f6258-c2aa-4905-9e25-078cc8a03787 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.804015 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 485/972] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:27.804643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.814013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:27.847653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.848605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.848929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.849197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.849944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:27.850244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.851869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8223d-0ea1-4bd8-8ea6-a1741879717b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.870275 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 488/973] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Sep 11 00:43:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:27.871632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d04d-b531-4181-b734-8c1715b4feca None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.871916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d04d-b531-4181-b734-8c1715b4feca None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.872187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887d04d-b531-4181-b734-8c1715b4feca None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.873366 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 489/974] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:27.875886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d04d-b531-4181-b734-8c1715b4feca None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:27.908624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:27.921820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfb7b1-8389-46e5-bdad-4a2aa07ab3cb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:27.956561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:27.957191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.957464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.957733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.964748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:27.966865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f45769e0-6fe4-434e-8d30-5e60c54a8bdb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:27.978803 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:27.979877 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:27.980827 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 486/975] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:27.981446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98193c78-ca31-48ff-8092-23830d827601 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:27.981446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98193c78-ca31-48ff-8092-23830d827601 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:27.981566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98193c78-ca31-48ff-8092-23830d827601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:27.986167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887d04d-b531-4181-b734-8c1715b4feca None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.000201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-98193c78-ca31-48ff-8092-23830d827601 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:28.001390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-98193c78-ca31-48ff-8092-23830d827601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:28.021249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.022106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.022538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.022943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.062277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.071207 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 487/976] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:28.076370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98193c78-ca31-48ff-8092-23830d827601 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.093213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.094663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.123261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.185206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98193c78-ca31-48ff-8092-23830d827601 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.206139 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 490/977] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:27 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:28.206994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.229078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.230219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.230688 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.231283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.232748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:28.233220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.235575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.272971 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 488/978] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:28.274110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.309891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49ce82-b5f7-48ca-a258-cf72b79dcca8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.346649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.351591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.353365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.354345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.354919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.355407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.356051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.357445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:28.366849 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:28.368054 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:28.368942 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 491/979] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:28.369788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.406878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee372ad1-84f9-4f35-89d8-aeb54bae3ab1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.443909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.444551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.444825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.445094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.455971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.457426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:28.476376 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:28.478018 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:28.479222 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:28.479222 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:28.479222 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:28.479222 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 489/980] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:28.479222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.493519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae117bd-eb40-43bf-99b7-ad07eade5c26 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.536454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.537072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.537345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.537614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.538272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.539751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.556063 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 492/981] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:43:28 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:28.557911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.596997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53892a05-be57-4e3c-ba34-111082f1798a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.606337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.629697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.644190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.644844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.645113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.645377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.646014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:28.646310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.647730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.653529 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 490/982] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:28.660080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.686732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88069021-e7e7-4364-95b8-6532925d5f75 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.725048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.725670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.725939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.726193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.726844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:28.727134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.728554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.741836 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 493/983] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:28.747590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.775813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b44583-7b06-4dac-937f-a3dac2bc5b76 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.809993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.810602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.810876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.811146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.811788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:28.812088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.813444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.818209 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 491/984] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:28.824865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.855604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.857982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3954072-a4e4-487e-9ffc-f0f51d3bb72e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.874932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:28.894302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.894931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.895222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.895492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.902862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.904365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:28.915768 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:28.916828 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:28.918442 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:28.921670 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 494/985] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:28.922420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:28.935752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a2a58-0859-4b33-972d-28635134b191 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:28.971512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:28.972371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:28.972636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:28.972898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:28.973602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] RBAC: Adding query filter params (interface=admin, service_id=ead204df1f11413987ef08994edb6c20, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:28.973913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:28.975285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:28.984876 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 492/986] 10.222.0.23 () {56 vars in 1249 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/endpoints?service_id=ead204df1f11413987ef08994edb6c20&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:28.986422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:29.026066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff2633-1950-44de-a218-1a9187cc2c20 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:29.063005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:29.063793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:29.064186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:29.064541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:29.065321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:29.065718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:29.067272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:29.084412 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 495/987] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:29.086231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:29.090675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:29.097786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6bb429-b030-4de9-bd8d-baae44b38b5c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:29.120823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:29.137090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f488c0a-9c94-4210-9389-e057ef739ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:29.137692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f488c0a-9c94-4210-9389-e057ef739ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:29.137963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f488c0a-9c94-4210-9389-e057ef739ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:29.138227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f488c0a-9c94-4210-9389-e057ef739ff3 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:29.138826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f488c0a-9c94-4210-9389-e057ef739ff3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:29.142560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f488c0a-9c94-4210-9389-e057ef739ff3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:29.150229 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 493/988] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:28 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:43:29.205911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf73d-5ba3-4720-bedf-48e198486d42 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:29.245675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:29.246407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:29.246761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:29.247066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:29.247918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] RBAC: Adding query filter params (role.id=4ddea9e8dc7642d0bca59749640626d7, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=8f67555ada824965b62e7026cfbab0bf) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:29.248290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:29.249829 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77913fd4-4419-4e3d-9915-2684e3f0528b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:29.263475 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 496/989] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:29 2022] GET /identity/v3/role_assignments?user.id=8f67555ada824965b62e7026cfbab0bf&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=4ddea9e8dc7642d0bca59749640626d7 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:30.709690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142087d-0b07-4e60-9725-907aa825869f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:30.710273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142087d-0b07-4e60-9725-907aa825869f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:30.710273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142087d-0b07-4e60-9725-907aa825869f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:30.711788 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 494/990] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:30.716827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:30.717241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:30.717317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:30.735631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:30.736864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:30.806040 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 497/991] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:31.066624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142087d-0b07-4e60-9725-907aa825869f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.177832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142087d-0b07-4e60-9725-907aa825869f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.217002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f4b08-d441-4d81-b374-870276415eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.217628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4b08-d441-4d81-b374-870276415eae None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.217901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4b08-d441-4d81-b374-870276415eae None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.218166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f4b08-d441-4d81-b374-870276415eae None admin] PATH_INFO: `/v3/services/compute` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.218747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4b08-d441-4d81-b374-870276415eae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.220139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f4b08-d441-4d81-b374-870276415eae None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-261f4b08-d441-4d81-b374-870276415eae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:31.229157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:31.230151 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:31.231025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:31.235348 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 495/992] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:31.236197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.265890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:31.300990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e1bfa-4780-4663-a2e3-c1fb7116d483 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.301268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e1bfa-4780-4663-a2e3-c1fb7116d483 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.301530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e1bfa-4780-4663-a2e3-c1fb7116d483 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.302910 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 496/993] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:31.309230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d36dc5-a0f0-4164-a58e-111776c7377c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.309591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d36dc5-a0f0-4164-a58e-111776c7377c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.309755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d36dc5-a0f0-4164-a58e-111776c7377c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.329605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-87d36dc5-a0f0-4164-a58e-111776c7377c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:31.331346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-87d36dc5-a0f0-4164-a58e-111776c7377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:31.347266 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b92e29-e8a7-4403-80fb-93130f1d14cf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.351389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:31.384476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.385085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.385364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.385628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.386256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:31.386553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.387946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:31.395261 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 498/994] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:31.398891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.410647 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 497/995] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:31.430747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154aadd-df06-41d7-a11f-c84598016d67 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.431024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154aadd-df06-41d7-a11f-c84598016d67 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.431332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154aadd-df06-41d7-a11f-c84598016d67 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.432662 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 498/996] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:31.438090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.438369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.438639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.459659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:31.460826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:31.502291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:31.510336 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec680a9-aebd-40d3-ab73-b2c3685e26b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.532017 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 499/997] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:31.543491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.544099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.544372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.544636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.545270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:31.545570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.546935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:31.556915 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 499/998] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:31.557896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.640269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:31.660248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57e9ed-95a0-4f78-b355-fc568fd44734 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.676508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.696569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.697182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.697453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.697719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.698559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] RBAC: Adding query filter params (interface=public, service_id=0a021e36dca04c6a8a1a06c0fcd424d8, region_id=RegionOne) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:31.698868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.700295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:31.705850 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 500/999] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/endpoints?service_id=0a021e36dca04c6a8a1a06c0fcd424d8&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:31.711810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.781702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81396fd-77db-4b0c-a5dc-aab828d63b87 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.814825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42b95a-8419-482b-a221-5ba4cc24c5c1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.816354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.816983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.817250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.817508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.818079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.819452 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:31.827997 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:31.829653 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 11 00:43:31.830885 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:31.834741 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 500/1000] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:31.835606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.849355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.849942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.850210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.850469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.851060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.852446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:31.858695 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 501/1001] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:43:31.859309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:31.871929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:31.873327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:31.957334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95e9cc-ac0e-438f-aa2f-e26b20e7f2b7 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.982565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946fb90-e776-4545-b507-7d1a60ee6cee None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:31.995642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:31.996301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:31.996570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:31.996837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] PATH_INFO: `/v3/services/image` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:31.997435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:31.998826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:32.008339 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:32.009429 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:32.010453 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 501/1002] 10.222.0.23 () {56 vars in 1105 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:32.015886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:32.024687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:32.025502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:32.025775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:32.026047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:32.026701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:32.027000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:32.028423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:32.034050 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 502/1003] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:32.039583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:32.116486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:32.120893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:32.134052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca36b75-c5f9-401a-a3fe-c75fd01c784c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:32.148534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0eef9c-1674-4874-8468-9d1897bfc655 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:32.172569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:32.173185 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:32.173483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:32.173780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:32.174409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] RBAC: Adding query filter params (name=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:32.174705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:32.176097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:32.181083 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 502/1004] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:32.186940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:32.188723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:32.189389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:32.189659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:32.189920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:32.190543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:32.190837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:32.192214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:32.197376 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 503/1005] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Sep 11 00:43:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:32.202889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:32.293472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce7908b-0ff6-4546-8211-573d20dd3b91 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:32.309432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f4792-22e6-4ac2-9d6a-aaa0afaaa3e2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:32.329088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:32.329806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:32.330097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:32.330359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:32.330991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] RBAC: Adding query filter params (type=image) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:32.331306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:32.332665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75415ee7-3bac-4bad-a77a-c547f9355280 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:32.337409 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 503/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:32.346886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfab69c-be9e-404c-997c-90303ed24a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:32.347539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab69c-be9e-404c-997c-90303ed24a5b None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:32.347808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab69c-be9e-404c-997c-90303ed24a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:32.348069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfab69c-be9e-404c-997c-90303ed24a5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:32.348751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab69c-be9e-404c-997c-90303ed24a5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:32.350132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfab69c-be9e-404c-997c-90303ed24a5b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:32.366063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:32.379750 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 504/1007] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:32 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:43:34.001328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aca4a-a2a1-4384-8001-0c2bcd1b82e5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.001965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aca4a-a2a1-4384-8001-0c2bcd1b82e5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.001965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835aca4a-a2a1-4384-8001-0c2bcd1b82e5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.003262 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 504/1008] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:34.008933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22a13b-73ea-4192-9497-b172af9e9a6a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.009382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22a13b-73ea-4192-9497-b172af9e9a6a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.009481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22a13b-73ea-4192-9497-b172af9e9a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.027815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ad22a13b-73ea-4192-9497-b172af9e9a6a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:34.028992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ad22a13b-73ea-4192-9497-b172af9e9a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:34.103724 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 505/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:34.367397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aca4a-a2a1-4384-8001-0c2bcd1b82e5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:34.388752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:34.490683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f464e-ffca-4144-81aa-3aac7b66b785 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.491029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f464e-ffca-4144-81aa-3aac7b66b785 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.491299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f464e-ffca-4144-81aa-3aac7b66b785 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.493162 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 506/1010] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:34.493300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835aca4a-a2a1-4384-8001-0c2bcd1b82e5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:34.498335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b229b-0215-4c7f-bcf2-cee3afa34e56 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.498614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b229b-0215-4c7f-bcf2-cee3afa34e56 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.498887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b229b-0215-4c7f-bcf2-cee3afa34e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.521060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-db3b229b-0215-4c7f-bcf2-cee3afa34e56 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:34.522535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db3b229b-0215-4c7f-bcf2-cee3afa34e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:34.530474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:34.531104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.531405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.531668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.532252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:34.533702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ccfdc7fc-f30f-4565-a058-25810f6c074c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:34.543123 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:34.547619 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:34.548625 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 505/1011] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:34.549460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed76eaa-631e-4d3c-852f-5ab935ee5f90 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.549460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed76eaa-631e-4d3c-852f-5ab935ee5f90 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.549460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed76eaa-631e-4d3c-852f-5ab935ee5f90 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.549460 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 506/1012] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:34.550556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed76eaa-631e-4d3c-852f-5ab935ee5f90 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:34.599727 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 507/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:34.601321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.601587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.601849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.623962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:34.625171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:34.633267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:34.659685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed76eaa-631e-4d3c-852f-5ab935ee5f90 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:34.677723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:34.696726 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 508/1014] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:34.699520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:34.700141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.700412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.700673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.701300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:34.701595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:34.702955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:34.712633 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 507/1015] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:34.713729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:34.812707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd20f6-31e4-4565-adfd-3df94cfe693e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:34.845868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:34.846466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:34.846732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:34.846995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:34.847646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:34.847939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:34.849295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:34.858768 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 509/1016] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Sep 11 00:43:34 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:34.859924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:34.864237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:34.920628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:34.966591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a39c38-af36-463b-9dfe-a811cf31b699 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:34.977274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1cecdb-1d91-4db8-87d0-b3faedb2dff0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:35.001753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:35.002361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:35.002633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:35.002896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:35.003623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:35.004982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:35.010395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:35.010994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:35.011320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:35.011588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:35.012228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:35.013565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:35.020583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:35.031220 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 508/1017] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:34 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:35.031414 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 510/1018] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:43:34 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:35.031414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:35.165960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50a490-674b-4c80-97ac-0c863f49a0c6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:35.170045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:35.212018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5e868-b0f1-4216-ba0e-7c43c223217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:35.212578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e868-b0f1-4216-ba0e-7c43c223217e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:35.212852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e868-b0f1-4216-ba0e-7c43c223217e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:35.213120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5e868-b0f1-4216-ba0e-7c43c223217e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:35.214314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5e868-b0f1-4216-ba0e-7c43c223217e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:35.215818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae5e868-b0f1-4216-ba0e-7c43c223217e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:35.219210 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 511/1019] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:35.223583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:35.335351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5beb9f2-821c-4630-bb7b-ee89b3863037 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:35.370019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd977bba-5756-4042-8518-068226180f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:35.370629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977bba-5756-4042-8518-068226180f13 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:35.370901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977bba-5756-4042-8518-068226180f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:35.371184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd977bba-5756-4042-8518-068226180f13 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:35.371879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977bba-5756-4042-8518-068226180f13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:35.373264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd977bba-5756-4042-8518-068226180f13 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:35.402665 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 509/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:43:35 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:37.216286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd8ae4-a824-4745-819d-5e4753bb955d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.216843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd8ae4-a824-4745-819d-5e4753bb955d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.216843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd8ae4-a824-4745-819d-5e4753bb955d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.218138 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 512/1021] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:37.224291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447b04e-6f3a-477f-a6b2-935d462e64a8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.224708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447b04e-6f3a-477f-a6b2-935d462e64a8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.224816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447b04e-6f3a-477f-a6b2-935d462e64a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.243701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2447b04e-6f3a-477f-a6b2-935d462e64a8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:37.245087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2447b04e-6f3a-477f-a6b2-935d462e64a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:37.291503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:37.308591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5c4f-afed-4621-bb70-297136b8d729 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.308846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5c4f-afed-4621-bb70-297136b8d729 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.309425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf5c4f-afed-4621-bb70-297136b8d729 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.311301 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 513/1022] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:37.317756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.318224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.318484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.324212 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 510/1023] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:37.337238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:37.338416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:37.404529 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 514/1024] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:37.582221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fea93-072e-497d-a186-c025a0cc59f9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.582563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fea93-072e-497d-a186-c025a0cc59f9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.582769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8fea93-072e-497d-a186-c025a0cc59f9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.584192 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 511/1025] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:37.587312 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:37.588941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.589215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.589479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.610784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:37.612142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:37.682769 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 512/1026] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:37.683996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:37.699228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94be3fc-e592-4f16-b714-ccc814930e66 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:37.700739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:37.735468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:37.736146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.736414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.736676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.737253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:37.738600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:37.748077 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:37.749076 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:37.750025 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 515/1027] 10.222.0.23 () {56 vars in 1105 bytes} [Sun Sep 11 00:43:37 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:37.755198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:37.789740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f388eeb-df83-4433-b915-ea368c358218 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:37.797412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:37.825296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:37.825926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.826189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.826450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.836467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:37.841113 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:37.842267 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:37.843398 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:37.847107 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 513/1028] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Sep 11 00:43:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:37.848389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:37.865857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ff978-4761-4b14-8952-acc0185b3cf3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:37.899691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:37.900289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.900548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.900816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.901424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] RBAC: Adding query filter params (name=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:37.901722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:37.903068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:37.908480 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 516/1029] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:37.913959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:37.946471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:37.957858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533e9ef-2d57-4e12-a476-e9759b3862ea None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:37.991637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:37.992239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:37.992511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:37.992778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:37.993432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:37.993729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:37.995112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.003153 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 514/1030] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:43:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:38.004281 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.014765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a372d5f-e2f6-44c9-b297-f91ca124684c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.032515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.048043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.048703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.048984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.049249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.049874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] RBAC: Adding query filter params (type=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:38.050172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.051593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.056431 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 517/1031] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:37 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:38.057076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.107108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c84f6-94c7-447f-a8d7-64f1f436c30c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.139600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.140467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.140714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.141014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.142259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.143742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.145326 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 515/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:38.146076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.151749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.159493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bd01a-04fc-4986-ab10-08fd663a221d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.194149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.194741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.195014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.195316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.206178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.207664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:38.222407 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:38.223978 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:38.224878 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:38.224878 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:38.224878 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:38.224878 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 518/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:38.224878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.242348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.248540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b51e0-4067-470d-9d38-d912d9d71053 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.280864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.281468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.281738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.282009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.282721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] RBAC: Adding query filter params (interface=public, service_id=a398d5be779e4c818d0427fff356b740, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:38.283024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.284420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.289285 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 516/1034] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/endpoints?service_id=a398d5be779e4c818d0427fff356b740&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:38.290035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.328497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c9647-3f77-40d6-89f5-554484c9551f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.364938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.365459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.365715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.365968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.366593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.368058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.385313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0f70d-e5d4-4123-8cc2-2f1f2b751f42 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.391581 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 519/1035] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:43:38 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:38.392531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.395150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.440820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.441946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.442223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.442489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.443560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:38.443869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.445795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.461359 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 517/1036] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:38.471223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.493961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.516528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d44f8-2212-4edc-a544-4e0b26bbc3c2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.555515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.556130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.556399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.556665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.557635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.559387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.572248 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 520/1037] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:43:38.603198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2958a82-52c1-4891-9594-042a460a141b None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.643261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.643948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.644230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.644501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.653138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.654700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:38.669140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:38.670204 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:38.671368 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:38.672069 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 518/1038] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:38.677781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.749137 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.789541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a20631-b7cb-4e7e-9a8d-582ae05def5b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.826229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.826839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.827109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.827398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:38.828116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:38.828417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:38.829857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:38.852423 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 521/1039] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:38.853369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:38.938624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:38.961231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da531ec1-cb49-4ff6-bb4a-ce2315f4c5af None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:38.995289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:38.995896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:38.996161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:38.996418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:39.065506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:39.066952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:39.149134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:39.175660 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 519/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 00:43:39.181362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:39.283457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad937b8-36e2-4ce5-87c8-5b2c99be4b5b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:39.307108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:39.319711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:39.320345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:39.320624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:39.320885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:39.331819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:39.333272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:39.347883 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:39.348920 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:39.349848 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:39.349848 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:39.349848 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:39.349848 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 522/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:39.356323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:39.457293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98012535-cc37-454b-b460-d864d38ee69f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:39.494783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:39.495377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:39.495643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:39.495987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:39.496547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:39.496839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:39.498197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:39.509833 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 520/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:39.515518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:39.614300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20535e7d-3c9f-45a3-b18d-97d8068bef58 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:39.647018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:39.647663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:39.647912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:39.648190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:39.654746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:39.656193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:39.667125 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:39.668139 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:39.669081 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:39.670236 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 523/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:39.674248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:39.720379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:39.783381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e934b6-bdc6-4aee-9ade-7d38cefac842 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:39.822417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:39.823030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:39.823335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:39.823605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:39.824301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:39.824602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:39.826037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4bc9e-273d-4f4c-b302-177ea95b9906 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:39.839844 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 521/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:39.845915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:39.884130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:39.950959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d15bc-9fc5-45d6-8bea-d4e1a93681d1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:39.985127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:39.985728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:39.986030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:39.986298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:39.987022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] RBAC: Adding query filter params (role.id=4ddea9e8dc7642d0bca59749640626d7, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=c150143af9144547ab3dc5a5d318d45f) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:39.987367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:39.988786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8d151-5c87-4ad7-bc5e-c40e6b58eb61 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:39.994306 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 524/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:39 2022] GET /identity/v3/role_assignments?user.id=c150143af9144547ab3dc5a5d318d45f&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=4ddea9e8dc7642d0bca59749640626d7 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:40.634865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1688b5-928b-4343-8a83-adfd15bca003 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:40.635467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1688b5-928b-4343-8a83-adfd15bca003 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:40.635467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1688b5-928b-4343-8a83-adfd15bca003 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:40.637032 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 522/1046] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:40.643102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f63146-02e7-4e22-87fa-c02bed576353 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:40.643700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f63146-02e7-4e22-87fa-c02bed576353 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:40.643780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f63146-02e7-4e22-87fa-c02bed576353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:40.664986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d8f63146-02e7-4e22-87fa-c02bed576353 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:40.666228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d8f63146-02e7-4e22-87fa-c02bed576353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:40.739880 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 525/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:40.765865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176de7b-d840-4f0e-96b2-592c8733d6a2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:40.766225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176de7b-d840-4f0e-96b2-592c8733d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:40.766562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176de7b-d840-4f0e-96b2-592c8733d6a2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:40.768048 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 523/1048] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:40.773386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:40.773678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:40.773919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:40.793831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:40.795019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:40.822105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:40.868766 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 526/1049] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:41.004713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176de7b-d840-4f0e-96b2-592c8733d6a2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.104610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.109849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176de7b-d840-4f0e-96b2-592c8733d6a2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.135527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.149698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.150277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.150551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.150822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.162346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:41.167159 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:41.168133 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:41.169007 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:41.173413 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 524/1050] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:41.174139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.247311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbb1d6-ecf0-4dd8-9afa-16fe3a892e90 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.284256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.284796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.284905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01cff8-0f4a-4bf7-91c1-ca237397dc2c None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.285078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.285320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] PATH_INFO: `/v3/services/image` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.285893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.287274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:41.295884 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:41.296989 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:41.297860 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 527/1051] 10.222.0.23 () {56 vars in 1105 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:41.302941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.323463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.324068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.324344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.324611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.325418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:41.325598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.326978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.329595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.345143 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 525/1052] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:41.345143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.370044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.411871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2d137-d394-48dc-bc22-76f0291b1e23 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.446282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.446890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.447165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.447465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.448089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] RBAC: Adding query filter params (name=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:41.448389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.449749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.449853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f336bbb-d54c-4e2e-9a41-4dd90366cf34 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.459478 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 528/1053] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:41.460504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.484090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.484720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.485094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.485372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] PATH_INFO: `/v3/users/623273fb33ba49b1932ad490e0694142` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.498956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] RBAC: Authorizing `identity:get_user(user_id=623273fb33ba49b1932ad490e0694142)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.500509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.502210 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 526/1054] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/users/623273fb33ba49b1932ad490e0694142 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:41.507877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.560146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.573326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f76f-0f94-4823-a501-39e3169c12ca None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.592894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.607301 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.608006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.608292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.608562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.609215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] RBAC: Adding query filter params (type=image) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:41.609509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.610907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.615773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af6ce2-0eea-42c0-98bc-6ece3d9f85fc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.620874 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 529/1055] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:41.621929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.650858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.651733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.651847 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.652040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.721756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.723775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.728605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2be70-2318-4dfc-979b-ca60e2243e4d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:41.765446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:41.766062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:41.766341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:41.766606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:41.767339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:41.768709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:41.798860 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 530/1056] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:43:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:41.818957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.841988 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 527/1057] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 00:43:41.848175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:41.860681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:41.968442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df049e31-5369-415f-aacb-d5f2a8ab5465 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:42.006149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837a5ad-7817-43ec-9079-7fc56ad0f3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:42.006793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837a5ad-7817-43ec-9079-7fc56ad0f3cf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.007070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837a5ad-7817-43ec-9079-7fc56ad0f3cf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.007380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837a5ad-7817-43ec-9079-7fc56ad0f3cf None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:42.008542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837a5ad-7817-43ec-9079-7fc56ad0f3cf None admin] RBAC: Authorizing `identity:get_project(project_id=77de43f62a404023a4ae30e7117ab498)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:42.010025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837a5ad-7817-43ec-9079-7fc56ad0f3cf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:42.012018 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 531/1058] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:43:41 2022] GET /identity/v3/projects/77de43f62a404023a4ae30e7117ab498 => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:42.018496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:42.123459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adab4c3-974f-4aaf-9b9e-a9e97679976d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:42.137835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a55fc7-712a-4047-a443-dc4d42a63d40 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.138106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a55fc7-712a-4047-a443-dc4d42a63d40 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.138391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a55fc7-712a-4047-a443-dc4d42a63d40 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:42.139901 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 532/1059] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:43:42.146290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.146564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.146828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:42.174658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:42.175215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:42.175832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.176106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.176371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:42.177108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=77de43f62a404023a4ae30e7117ab498, user.id=623273fb33ba49b1932ad490e0694142) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:42.177408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:42.177955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:42.178940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:42.185862 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 528/1060] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:42 2022] GET /identity/v3/role_assignments?user.id=623273fb33ba49b1932ad490e0694142&scope.project.id=77de43f62a404023a4ae30e7117ab498&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:42.271225 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 533/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:42.534394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:42.584089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:42.639535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9538-2230-41e0-b66e-292bc20ba9c1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:42.673570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:42.674247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.674506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.674691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:42.685619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:42.690323 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:42.691303 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:42.696173 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 529/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Sep 11 00:43:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:42.697569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:42.699824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:42.799734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19010d50-291f-4f76-a226-99c42c67bfd2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:42.833327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:42.833926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.834191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.834450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:42.835099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:42.835420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:42.836771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:42.844960 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 534/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:43:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:42.850605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:42.950458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c851f-412e-4087-859d-85fcc45bdda9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:42.956385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:42.988595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:42.989223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:42.989490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:42.989750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.000490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.001979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:43.015750 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:43.016803 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:43.017779 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:43.017779 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:43.017779 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:43.017779 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 530/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:43.024214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:43.089388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:43.129956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09ef5f-638e-4d8a-b4e7-79328ed88ca4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:43.168635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:43.169240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.169507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.169770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.170396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:43.170689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.172069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.184402 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 535/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:43.190221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:43.287768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475be9c-77e6-4aa3-bafd-19927580d1b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:43.320878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:43.321495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.321763 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.322019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.329531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.330984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:43.341870 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:43.342864 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:43.343890 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:43.347283 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 531/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:43:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:43.349114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:43.453266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e93d3-3525-48bc-a743-34ec49b0c5e4 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:43.476870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:43.489229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:43.489839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.490110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.490375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.491095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:43.491425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.492843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.510622 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 536/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:43:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:43.516314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:43.535470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:43.619327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bba911-876d-4cb1-b5cd-1ae57282afcf None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:43.653033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:43.653657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.653926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.654191 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.664752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.666185 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:43.680222 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:43.681650 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:43.683705 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:43.683705 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:43.683705 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:43.686418 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 532/1068] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:43.687490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:43.787349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c289b7-c53f-446a-b683-fa15e079a787 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:43.821514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:43.822197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.822416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.822629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.823292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:43.823591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.824937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40406237-8cea-4622-8778-bb08081fbc43 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.836482 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 537/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:43.842030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:43.915379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:43.951425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2078-e9fb-4862-aa18-7076fce7e556 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:43.986028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:43.986654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.986925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.987212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.993873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:43.995339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:43.997059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d63ccb-0f69-4947-b0fd-716530f8e1de None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:43.997550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d63ccb-0f69-4947-b0fd-716530f8e1de None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:43.998019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d63ccb-0f69-4947-b0fd-716530f8e1de None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:43.999557 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 538/1070] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:44.004798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.005074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.005342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:44.006732 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:44.007634 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:44.008554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:44.011333 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 533/1071] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:44.013398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.025974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:44.027298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:44.065290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:44.099289 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 539/1072] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:44.115163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b9b49-ec9e-4182-a112-90606303087e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:44.145614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:44.156560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:44.156560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.156560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.156560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.156940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:44.156940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:44.158399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5ede3-a59a-4053-90a1-27063e631f6a None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:44.175942 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 534/1073] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:44.177547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.286427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19745-25e8-4a79-90c6-408bad9d190f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:44.320612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:44.321271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.321541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.321807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/c150143af9144547ab3dc5a5d318d45f/roles/4ddea9e8dc7642d0bca59749640626d7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.324080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=c150143af9144547ab3dc5a5d318d45f, role_id=4ddea9e8dc7642d0bca59749640626d7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:44.325819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:44.327333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c07fd-0b95-415f-b712-bb6ea0ce3ff6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.327622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c07fd-0b95-415f-b712-bb6ea0ce3ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.327918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c07fd-0b95-415f-b712-bb6ea0ce3ff6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.329361 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 535/1074] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:44.334252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.334525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.334795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.342500 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 540/1075] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:44 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/c150143af9144547ab3dc5a5d318d45f/roles/4ddea9e8dc7642d0bca59749640626d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:43:44.364905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:44.365280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.366104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:44.401874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:44.450972 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 536/1076] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:43:44.480558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4968943-b6b5-40cf-b3d8-470bd5ba27b0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:44.518171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:44.518797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.519075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.519963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.589921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:44.591430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:44.622850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:44.699222 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 541/1077] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:44.705732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.709612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.810383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2480bd-3671-4137-8a1f-57b6a5d4a5f9 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:44.812687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa4e74-ecb5-438b-880c-fe81018f54cf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:44.831312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:44.842444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:44.848243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:44.848872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.849148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.849417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.855561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:44.856554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:44.857274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:44.857722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:44.859549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:44.864362 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:44.865312 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:44.866220 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 542/1078] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:44.870566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:44.871428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.871994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:44.888511 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:44.889519 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:44.890444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:44.890444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:44.890444 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:44.890444 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 537/1079] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:44.895655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:44.987947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6413dbf-4aef-41db-98d9-53d208a00fc7 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:44.997445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085635f-303b-41fb-9686-2f6530fa67aa None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:45.024748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:45.025370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:45.025825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:45.025934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:45.026544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:45.026862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:45.028251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:45.034685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:45.035541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:45.035991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:45.036428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:45.037132 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 543/1080] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:45.037431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:45.037857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:45.039370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:45.043006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:45.052661 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 538/1081] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:45.105848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:45.168096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef9f04-9c98-410e-b5e7-a4f46b1fd3ac None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:45.207015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:45.207695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:45.207965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:45.208229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] PATH_INFO: `/v3/users/623273fb33ba49b1932ad490e0694142` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:45.209418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] RBAC: Authorizing `identity:get_user(user_id=623273fb33ba49b1932ad490e0694142)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:45.210863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:45.212698 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 544/1082] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:45 2022] GET /identity/v3/users/623273fb33ba49b1932ad490e0694142 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:45.218481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:45.254198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:45.332309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ad54c-419f-45fc-accc-306fd9ec8988 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:45.367096 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c555199-3c85-4665-a5ec-99c02ae2504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:45.367759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555199-3c85-4665-a5ec-99c02ae2504e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:45.368037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555199-3c85-4665-a5ec-99c02ae2504e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:45.368306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c555199-3c85-4665-a5ec-99c02ae2504e None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:45.369442 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c555199-3c85-4665-a5ec-99c02ae2504e None admin] RBAC: Authorizing `identity:get_project(project_id=77de43f62a404023a4ae30e7117ab498)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:45.370876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c555199-3c85-4665-a5ec-99c02ae2504e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:45.372874 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 539/1083] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:43:45 2022] GET /identity/v3/projects/77de43f62a404023a4ae30e7117ab498 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:45.379111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:45.479365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fcf4e-ba3c-4e4d-9ff6-f75c7e8c3aba None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:45.501464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:45.513561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c594f1e-13d9-4362-9362-30eb1056e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:45.514151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c594f1e-13d9-4362-9362-30eb1056e601 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:45.514423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c594f1e-13d9-4362-9362-30eb1056e601 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:45.514696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c594f1e-13d9-4362-9362-30eb1056e601 None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498/users/623273fb33ba49b1932ad490e0694142/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:45.516654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c594f1e-13d9-4362-9362-30eb1056e601 None admin] RBAC: Authorizing `identity:create_grant(project_id=77de43f62a404023a4ae30e7117ab498, user_id=623273fb33ba49b1932ad490e0694142, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:45.518095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c594f1e-13d9-4362-9362-30eb1056e601 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:45.532188 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 545/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:45 2022] PUT /identity/v3/projects/77de43f62a404023a4ae30e7117ab498/users/623273fb33ba49b1932ad490e0694142/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:43:46.524044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de7679-77bb-4390-bb20-d3b705350f32 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:46.524565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de7679-77bb-4390-bb20-d3b705350f32 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:46.524565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de7679-77bb-4390-bb20-d3b705350f32 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:46.525959 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 540/1085] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:46.531394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:46.531783 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:46.531970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:46.551209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:46.552420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:46.621004 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 546/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:46.886057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de7679-77bb-4390-bb20-d3b705350f32 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:46.983291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:46.988926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de7679-77bb-4390-bb20-d3b705350f32 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.022543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.023160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.023454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.023716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.033757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:47.038553 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:47.043558 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:47.044807 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 541/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Sep 11 00:43:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:47.045762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:47.149207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3dedcd-10a5-4caa-b7ad-272994205b3a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.162540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:47.184472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.185077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.185348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.185613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.186277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:47.186571 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:47.187981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:47.195860 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 547/1088] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:43:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:47.201851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:47.302985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bb53f-1630-443c-ae66-ebf2d6995ef0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.338203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.338831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.339101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.339413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.349317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:47.350759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:47.365001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:47.366020 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:47.366942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:47.366942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:47.366942 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:47.366942 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 542/1089] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:47.371992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:47.473477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13c52a-93b4-49ec-93c3-b266901088dc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.507706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.508305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.508577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.508845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.509484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:47.509780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:47.511136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55507dd-2252-4764-95a3-1a75093e857d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:47.523918 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 548/1090] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:47.529889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:47.544332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:47.633531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc06b2-d25e-4377-b468-633c27c24362 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.645229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9b86-8cf1-4cec-af5c-b23bc1c52b65 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.645747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9b86-8cf1-4cec-af5c-b23bc1c52b65 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.645747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9b86-8cf1-4cec-af5c-b23bc1c52b65 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.647145 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 549/1091] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:47.652257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.652529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.652792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.668108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.668729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.669002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.669275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.672095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:47.673285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:47.677728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:47.679146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:43:47.690976 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:47.692001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:47.692993 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 543/1092] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Sep 11 00:43:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:47.697843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:47.700341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:47.743873 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 550/1093] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:47.764233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:47.793655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a15b0-69f7-4217-a56d-3a8dbcb95cb8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.827604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.828190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.828461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.828724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:47.829489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:47.829786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:47.831205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:47.848229 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 544/1094] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Sep 11 00:43:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:47.854173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:47.954018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15435af-a02c-4ae9-8af6-79540cce44d1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:47.988222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:47.988822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:47.989095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:47.989330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.011410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.023858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.063323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.064802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.113698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35cce0-27f1-4ba6-ae30-3e7ca9317733 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.140122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.150957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.151645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.151911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.152173 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.164341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:43:48.166491 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 551/1095] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:43:48.169072 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:48.170083 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:43:48.171001 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 545/1096] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:43:48.172578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.175908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.231110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.269010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb7fc3-53f5-4c32-ace9-248d2c8ff343 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.281396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f94761-c90c-4275-9c49-400c22176bef None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.303730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.304378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.304628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.304888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.315716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.317147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.317273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.317881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.318157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.318431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.319095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:48.319643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.320791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.329046 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 546/1097] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:43:48.331192 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:48.336999 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:43:48.338090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:43:48.338090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:43:48.338090 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:48.338090 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 552/1098] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:48.338334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.338447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.387848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.437511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e2300-6ae3-41af-a6a1-806dcd8ae818 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.442294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6e8c9-9655-4eff-88ae-9be774b7d025 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.457621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.470043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.470690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.470964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.471254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] PATH_INFO: `/v3/users/623273fb33ba49b1932ad490e0694142` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.472419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] RBAC: Authorizing `identity:get_user(user_id=623273fb33ba49b1932ad490e0694142)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.473845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.475564 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 547/1099] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/users/623273fb33ba49b1932ad490e0694142 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:48.477420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.478152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.478429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.478697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.479369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:48.479665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.481096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.481239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.492833 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 553/1100] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:48.498375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.584156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6cd8a-d358-4a25-9def-fe3a0b52d6fb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.593143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ddb50-88ef-4b2d-8c3b-78df3426663c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.601304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.618531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.619196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.619495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.619742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.627850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.628486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.628762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.629029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.636123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.637571 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:43:48.649138 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:43:48.650683 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:43:48.651926 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 554/1101] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:43:48.656155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.688544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.690040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.693272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.756959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba4c3e-8b2d-4ef2-9bf7-e94c4e538684 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.786838 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 548/1102] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 00:43:48.792256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.792803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.792900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.793156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.793393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.794105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:48.794392 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.795878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.808635 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 555/1103] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:48.811034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.814906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.889414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c06686-d456-4b9d-9770-5ab8a941ba35 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.908043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:48.913945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67590cf7-05e8-47d3-88e6-83e779127c83 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:48.924756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.925383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.925652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.925919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.927066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=77de43f62a404023a4ae30e7117ab498)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.928584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.930227 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 549/1104] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/projects/77de43f62a404023a4ae30e7117ab498 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:48.936904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:48.948260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:48.948859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:48.949116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:48.949390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:48.950103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] RBAC: Adding query filter params (role.id=4ddea9e8dc7642d0bca59749640626d7, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=c150143af9144547ab3dc5a5d318d45f) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:48.950402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:48.951844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb04ca0-6c0b-466e-a066-249fbf0c6c68 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:48.957615 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 556/1105] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/role_assignments?user.id=c150143af9144547ab3dc5a5d318d45f&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=4ddea9e8dc7642d0bca59749640626d7 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:49.025452 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:49.042098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ac900-e531-46a9-afda-abe806331ef8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:49.079259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:49.079904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:49.080173 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:49.080438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:49.081175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=77de43f62a404023a4ae30e7117ab498, user.id=623273fb33ba49b1932ad490e0694142) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:49.081472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:49.082924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171249e-245d-4420-889f-9fd78f0375e4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:49.089802 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 550/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:48 2022] GET /identity/v3/role_assignments?user.id=623273fb33ba49b1932ad490e0694142&scope.project.id=77de43f62a404023a4ae30e7117ab498&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:51.254343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea54c-a5c2-437a-b002-24cfd5686723 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:51.254886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea54c-a5c2-437a-b002-24cfd5686723 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:51.254886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ea54c-a5c2-437a-b002-24cfd5686723 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:51.256251 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 557/1107] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:51.261810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:51.262193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:51.262346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:51.281092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:51.282275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:51.365942 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 551/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:51.633966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ea54c-a5c2-437a-b002-24cfd5686723 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:51.746807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ea54c-a5c2-437a-b002-24cfd5686723 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:51.780772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4742386-5ff5-4ea0-bab6-f110c566c985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:51.781401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4742386-5ff5-4ea0-bab6-f110c566c985 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:51.781676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4742386-5ff5-4ea0-bab6-f110c566c985 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:51.781947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4742386-5ff5-4ea0-bab6-f110c566c985 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:51.783102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4742386-5ff5-4ea0-bab6-f110c566c985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:51.784565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4742386-5ff5-4ea0-bab6-f110c566c985 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:51.786126 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 558/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:51.795480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:51.894767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:51.916562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a046-9f7c-46fa-ac60-8363578dd1ba None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:51.951634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead87e6-79d1-4606-ac87-36eb7c0c3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:51.952231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead87e6-79d1-4606-ac87-36eb7c0c3e9b None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:51.952502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead87e6-79d1-4606-ac87-36eb7c0c3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:51.952766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead87e6-79d1-4606-ac87-36eb7c0c3e9b None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:51.953404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead87e6-79d1-4606-ac87-36eb7c0c3e9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:51.954976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead87e6-79d1-4606-ac87-36eb7c0c3e9b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:51.988473 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 552/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:43:51 2022] POST /identity/v3/users => generated 286 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:43:54.155119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41da8b-3dd3-45ad-b89f-db977c5fb497 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:54.155754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41da8b-3dd3-45ad-b89f-db977c5fb497 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:54.155754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae41da8b-3dd3-45ad-b89f-db977c5fb497 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:54.157021 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 559/1111] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:54.162520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:54.162956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:54.163055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:54.183603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:54.184803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:54.254224 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 553/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:54.533449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41da8b-3dd3-45ad-b89f-db977c5fb497 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:54.565793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:54.644502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae41da8b-3dd3-45ad-b89f-db977c5fb497 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:54.679628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:54.680248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:54.680515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:54.680780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] PATH_INFO: `/v3/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:54.682188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] RBAC: Authorizing `identity:get_role(role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:54.683936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:54.685223 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 560/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:54 2022] GET /identity/v3/roles/9056804b78cc417c8a175569edc8c78a => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:54.694109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:54.759907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:54.800195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fb809c-58a7-46b1-8320-2ba405d009f2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:54.835881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:54.836497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:54.836766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:54.837030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] PATH_INFO: `/v3/users/ffe5bc530db54b37840f13a516d3f95a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:54.850673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] RBAC: Authorizing `identity:get_user(user_id=ffe5bc530db54b37840f13a516d3f95a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:54.852145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:54.853740 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 554/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:54 2022] GET /identity/v3/users/ffe5bc530db54b37840f13a516d3f95a => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:43:54.859973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:54.953943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:54.960234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9cc57-6bdd-447a-8ecc-fce14ad7472a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:54.994108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:54.994671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:54.994968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:54.995267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:55.063661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:55.065017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:55.159228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:55.166433 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 561/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:43:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:43:55.172583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:55.274327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be615fd3-d699-489e-a8c3-1bed40bf2e8d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:55.298862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:55.311838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41867a7a-b933-47e7-ab27-83c6f283b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:55.312454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867a7a-b933-47e7-ab27-83c6f283b811 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:55.312726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867a7a-b933-47e7-ab27-83c6f283b811 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:55.312989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41867a7a-b933-47e7-ab27-83c6f283b811 None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:55.314215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867a7a-b933-47e7-ab27-83c6f283b811 None admin] RBAC: Authorizing `identity:get_project(project_id=77de43f62a404023a4ae30e7117ab498)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:55.315614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41867a7a-b933-47e7-ab27-83c6f283b811 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:55.317222 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 555/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:43:55 2022] GET /identity/v3/projects/77de43f62a404023a4ae30e7117ab498 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:55.323764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:55.422618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556fb79-c5ed-4944-add1-ad30a2010b74 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:55.457199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:55.457778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:55.458047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:55.458309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:55.459021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] RBAC: Adding query filter params (role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=77de43f62a404023a4ae30e7117ab498, user.id=ffe5bc530db54b37840f13a516d3f95a) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:43:55.459351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:55.460775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc265f0f-76a2-4a55-afeb-60fa9c4c6dbb None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:55.466156 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 562/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:43:55 2022] GET /identity/v3/role_assignments?user.id=ffe5bc530db54b37840f13a516d3f95a&scope.project.id=77de43f62a404023a4ae30e7117ab498&role.id=9056804b78cc417c8a175569edc8c78a => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:57.609359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255b79-2e38-409a-bd43-9d13a1c17ff2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:57.609916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255b79-2e38-409a-bd43-9d13a1c17ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:57.609916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1255b79-2e38-409a-bd43-9d13a1c17ff2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:57.611581 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 556/1118] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:43:57.617002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:57.617464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:57.617557 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:57.636976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:43:57.638150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:43:57.703042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:57.715710 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 563/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:43:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:43:57.997233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255b79-2e38-409a-bd43-9d13a1c17ff2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:58.105190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1255b79-2e38-409a-bd43-9d13a1c17ff2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:58.142187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:58.142821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:58.143087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:58.143382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] PATH_INFO: `/v3/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:58.145316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] RBAC: Authorizing `identity:get_role(role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:58.147549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:58.149636 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 557/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:57 2022] GET /identity/v3/roles/9056804b78cc417c8a175569edc8c78a => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:58.156356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:58.256422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f5d7d-6070-4f29-8fc8-18c7cf8e5289 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:58.290167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:58.290780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:58.291054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:58.291349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] PATH_INFO: `/v3/users/ffe5bc530db54b37840f13a516d3f95a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:58.292504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] RBAC: Authorizing `identity:get_user(user_id=ffe5bc530db54b37840f13a516d3f95a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:58.293932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:58.295566 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 564/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:43:58 2022] GET /identity/v3/users/ffe5bc530db54b37840f13a516d3f95a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:58.301820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:58.366052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:58.410091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea67fc-2103-476d-87ce-f36be6193d79 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:58.446798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cea4f-be5d-4850-9bff-d01d279690b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:58.447537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cea4f-be5d-4850-9bff-d01d279690b4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:58.447803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cea4f-be5d-4850-9bff-d01d279690b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:58.448065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cea4f-be5d-4850-9bff-d01d279690b4 None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:58.449265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cea4f-be5d-4850-9bff-d01d279690b4 None admin] RBAC: Authorizing `identity:get_project(project_id=77de43f62a404023a4ae30e7117ab498)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:58.450686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cea4f-be5d-4850-9bff-d01d279690b4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:58.452315 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 558/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:43:58 2022] GET /identity/v3/projects/77de43f62a404023a4ae30e7117ab498 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:43:58.459648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:43:58.521421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:43:58.564233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf80f99-20a9-436f-adff-287e4a76e217 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:43:58.598667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192c245-5885-4d55-9004-4967380b085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:43:58.599340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c245-5885-4d55-9004-4967380b085c None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:43:58.599599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c245-5885-4d55-9004-4967380b085c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:43:58.599866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192c245-5885-4d55-9004-4967380b085c None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498/users/ffe5bc530db54b37840f13a516d3f95a/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:43:58.602166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c245-5885-4d55-9004-4967380b085c None admin] RBAC: Authorizing `identity:create_grant(project_id=77de43f62a404023a4ae30e7117ab498, user_id=ffe5bc530db54b37840f13a516d3f95a, role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:43:58.603649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192c245-5885-4d55-9004-4967380b085c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:43:58.618835 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 565/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:43:58 2022] PUT /identity/v3/projects/77de43f62a404023a4ae30e7117ab498/users/ffe5bc530db54b37840f13a516d3f95a/roles/9056804b78cc417c8a175569edc8c78a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:44:00.736805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a080-7cdf-4772-97e4-2703c970d37a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:00.737707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a080-7cdf-4772-97e4-2703c970d37a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:00.737707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37a080-7cdf-4772-97e4-2703c970d37a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:00.739860 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 559/1124] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:00.748793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:00.749235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:00.749335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:00.774724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:00.775940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:00.846127 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 566/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:01.111797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a080-7cdf-4772-97e4-2703c970d37a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:01.231745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37a080-7cdf-4772-97e4-2703c970d37a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:01.259629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:01.269113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:01.269733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:01.270002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:01.270262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] PATH_INFO: `/v3/roles/9056804b78cc417c8a175569edc8c78a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:01.271705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] RBAC: Authorizing `identity:get_role(role_id=9056804b78cc417c8a175569edc8c78a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:01.273069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:01.274389 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 560/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:01 2022] GET /identity/v3/roles/9056804b78cc417c8a175569edc8c78a => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:01.281093 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:01.334422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:01.388742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ffd85c-83b0-436b-a466-2e0e168fd76a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:01.423827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:01.424442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:01.424717 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:01.424980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] PATH_INFO: `/v3/users/ffe5bc530db54b37840f13a516d3f95a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:01.426218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] RBAC: Authorizing `identity:get_user(user_id=ffe5bc530db54b37840f13a516d3f95a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:01.427705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:01.429263 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 567/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:01 2022] GET /identity/v3/users/ffe5bc530db54b37840f13a516d3f95a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:44:01.435869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:01.544477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29037e97-d54c-4c10-ba96-cdaee67123ef None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:01.597516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:01.598145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:01.598411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:01.598674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:01.685404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:01.693590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:01.695083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:01.796454 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 561/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:44:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:44:01.803489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:01.906968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd7f1c-06fe-4b5c-8591-e53458240a2b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:01.924124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:01.943224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfaa76-3492-4a47-9f85-4a780c12bbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:01.943920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfaa76-3492-4a47-9f85-4a780c12bbe0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:01.944192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfaa76-3492-4a47-9f85-4a780c12bbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:01.944459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfaa76-3492-4a47-9f85-4a780c12bbe0 None admin] PATH_INFO: `/v3/projects/77de43f62a404023a4ae30e7117ab498` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:01.945576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfaa76-3492-4a47-9f85-4a780c12bbe0 None admin] RBAC: Authorizing `identity:get_project(project_id=77de43f62a404023a4ae30e7117ab498)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:01.947034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfaa76-3492-4a47-9f85-4a780c12bbe0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:01.948804 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 568/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:01 2022] GET /identity/v3/projects/77de43f62a404023a4ae30e7117ab498 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:01.956043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:02.063851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:02.070250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697b8ba-417e-43cc-8023-5b5a1c4b7d67 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:02.104066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:02.104620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:02.104915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:02.105174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:02.105894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] RBAC: Adding query filter params (role.id=9056804b78cc417c8a175569edc8c78a, scope.project.id=77de43f62a404023a4ae30e7117ab498, user.id=ffe5bc530db54b37840f13a516d3f95a) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:02.106196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:02.107675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc7c4a-3539-4147-9d0a-d1b4218e3000 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:02.113805 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 562/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:44:01 2022] GET /identity/v3/role_assignments?user.id=ffe5bc530db54b37840f13a516d3f95a&scope.project.id=77de43f62a404023a4ae30e7117ab498&role.id=9056804b78cc417c8a175569edc8c78a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:04.238350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434542-ac37-42a0-8f31-97c057a66dad None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:04.238901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434542-ac37-42a0-8f31-97c057a66dad None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:04.238901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f434542-ac37-42a0-8f31-97c057a66dad None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:04.240257 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 569/1131] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:04.246707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:04.247449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:04.247625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:04.269631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:04.270824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:04.342756 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 563/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:04.614152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f434542-ac37-42a0-8f31-97c057a66dad None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:04.731800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f434542-ac37-42a0-8f31-97c057a66dad None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:04.766211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a07fa-6cdd-40df-8d81-6dcd75ffbfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:04.766833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a07fa-6cdd-40df-8d81-6dcd75ffbfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:04.767104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a07fa-6cdd-40df-8d81-6dcd75ffbfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:04.767400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a07fa-6cdd-40df-8d81-6dcd75ffbfc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:04.768653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a07fa-6cdd-40df-8d81-6dcd75ffbfc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:04.770082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a07fa-6cdd-40df-8d81-6dcd75ffbfc7 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:04.771698 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 570/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:44:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:04.780441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:04.878199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:04.889412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226e0f7-d4f8-458d-8090-c3be94fcb06b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:04.924313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a334e7f-29f9-4f7c-8e5f-4ab2f04ef3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:04.924897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a334e7f-29f9-4f7c-8e5f-4ab2f04ef3c8 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:04.925162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a334e7f-29f9-4f7c-8e5f-4ab2f04ef3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:04.925422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a334e7f-29f9-4f7c-8e5f-4ab2f04ef3c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:04.926064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a334e7f-29f9-4f7c-8e5f-4ab2f04ef3c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:04.927520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a334e7f-29f9-4f7c-8e5f-4ab2f04ef3c8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:04.957714 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 564/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:44:04 2022] POST /identity/v3/projects => generated 311 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:44:07.125395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b337e-f978-405e-aa20-f607b33367e4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:07.126376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b337e-f978-405e-aa20-f607b33367e4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:07.126376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b337e-f978-405e-aa20-f607b33367e4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:07.128491 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 571/1135] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:07.135268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:07.135715 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:07.135810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:07.163089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:07.164378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:07.233816 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 565/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:07.520593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b337e-f978-405e-aa20-f607b33367e4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:07.579830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:07.631554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b337e-f978-405e-aa20-f607b33367e4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:07.685356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22de2199-fbf6-4b85-a2d2-85865565a2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:07.685997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2199-fbf6-4b85-a2d2-85865565a2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:07.686269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2199-fbf6-4b85-a2d2-85865565a2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:07.686533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22de2199-fbf6-4b85-a2d2-85865565a2a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:07.687905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de2199-fbf6-4b85-a2d2-85865565a2a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:07.689595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22de2199-fbf6-4b85-a2d2-85865565a2a1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:07.691212 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 572/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:44:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:07.698368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:07.791388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:07.821932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762af69-327c-4f2d-b9e1-4d9366e23901 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:07.855615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818545c-f7d9-491e-929e-dc379fa21d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:07.856210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818545c-f7d9-491e-929e-dc379fa21d70 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:07.856484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818545c-f7d9-491e-929e-dc379fa21d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:07.856752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818545c-f7d9-491e-929e-dc379fa21d70 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:07.857391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818545c-f7d9-491e-929e-dc379fa21d70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:07.858819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818545c-f7d9-491e-929e-dc379fa21d70 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:07.886461 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 566/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:44:07 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:44:10.066614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef039458-a711-4509-b1b9-fc4d9c95655c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:10.067584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef039458-a711-4509-b1b9-fc4d9c95655c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:10.067584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef039458-a711-4509-b1b9-fc4d9c95655c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:10.069725 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 573/1139] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:10.076037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:10.076467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:10.076582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:10.102861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:10.104097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:10.191275 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 567/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:10.454502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef039458-a711-4509-b1b9-fc4d9c95655c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:10.559500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef039458-a711-4509-b1b9-fc4d9c95655c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:10.588868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:10.611848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:10.612465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:10.612735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:10.613000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:10.625179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:44:10.629947 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:10.630997 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:44:10.632108 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 574/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:44:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:10.637505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:10.700629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:10.741959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa879f6b-f722-4e83-9483-e4767bc1f7b8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:10.785703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:10.786300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:10.786570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:10.786833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:10.787527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:10.787828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:10.789224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:10.797910 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 568/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:44:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:10.804245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:10.929289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ef3aa-f665-4ef6-9d2d-ca371db9df9d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:10.969225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:10.970052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:10.970339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:10.970601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] PATH_INFO: `/v3/users/60a9c49cd5234600b8ec0f3e83dd5852` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:10.984630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] RBAC: Authorizing `identity:get_user(user_id=60a9c49cd5234600b8ec0f3e83dd5852)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:10.986075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:10.987722 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 575/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:10 2022] GET /identity/v3/users/60a9c49cd5234600b8ec0f3e83dd5852 => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:10.993914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:11.099403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a89e2a1-896b-4b4a-a60a-7cba3f241c4f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:11.120166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:11.137225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:11.137812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:11.138080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:11.138342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:11.208304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:11.209760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:11.307252 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 569/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:44:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:44:11.313904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:11.357922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:11.418713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bcabb1-acc8-4cf7-b246-4116087ab621 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:11.455859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53434448-13c3-4001-8e3c-2b2a7b7887b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:11.456512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434448-13c3-4001-8e3c-2b2a7b7887b0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:11.456750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434448-13c3-4001-8e3c-2b2a7b7887b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:11.457033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53434448-13c3-4001-8e3c-2b2a7b7887b0 None admin] PATH_INFO: `/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:11.458181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53434448-13c3-4001-8e3c-2b2a7b7887b0 None admin] RBAC: Authorizing `identity:get_project(project_id=5661ed33e05e4ac88b1efc9eb2b9f8cd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:11.459860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53434448-13c3-4001-8e3c-2b2a7b7887b0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:11.461498 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 576/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:11 2022] GET /identity/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:44:11.468795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:11.494029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:11.575505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6c6cb-d306-4335-922d-ec7a077a5227 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:11.609641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:11.610224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:11.610494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:11.610746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:11.611487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=5661ed33e05e4ac88b1efc9eb2b9f8cd, user.id=60a9c49cd5234600b8ec0f3e83dd5852) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:11.611791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:11.613191 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a38fe-a4fb-409a-aecf-add35c194218 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:11.619125 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 570/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:44:11 2022] GET /identity/v3/role_assignments?user.id=60a9c49cd5234600b8ec0f3e83dd5852&scope.project.id=5661ed33e05e4ac88b1efc9eb2b9f8cd&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:13.726434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eae2fb-06aa-432c-926e-5df74e3e3e9c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:13.727462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eae2fb-06aa-432c-926e-5df74e3e3e9c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:13.727462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eae2fb-06aa-432c-926e-5df74e3e3e9c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:13.729388 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 577/1147] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:13.738078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:13.738495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:13.738649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:13.762311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:13.763521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:13.839715 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 571/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:14.105412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eae2fb-06aa-432c-926e-5df74e3e3e9c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:14.223447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eae2fb-06aa-432c-926e-5df74e3e3e9c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:14.230627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:14.257021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:14.257630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:14.257899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:14.258159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:14.268037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:44:14.272724 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:14.273845 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:44:14.274706 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 578/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:44:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:14.280629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:14.302766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:14.387390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f168dfa-7f12-4e6b-8009-3ccad5a25e5c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:14.424434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:14.425022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:14.425294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:14.425557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:14.426213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:14.426512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:14.427901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:14.437131 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 572/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:44:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:14.443949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:14.555712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6db494-c42a-4329-91d4-4e3dca372966 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:14.591118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:14.591744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:14.592020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:14.592282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] PATH_INFO: `/v3/users/60a9c49cd5234600b8ec0f3e83dd5852` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:14.593530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] RBAC: Authorizing `identity:get_user(user_id=60a9c49cd5234600b8ec0f3e83dd5852)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:14.596189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:14.597759 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 579/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:14 2022] GET /identity/v3/users/60a9c49cd5234600b8ec0f3e83dd5852 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:14.606811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:14.695056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:14.728564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5dd41-ee50-4441-8b6a-f37abb4397d2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:14.764269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ee192-deb1-4cfd-af49-f42aaacd65a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:14.764863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee192-deb1-4cfd-af49-f42aaacd65a6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:14.765138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee192-deb1-4cfd-af49-f42aaacd65a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:14.765404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee192-deb1-4cfd-af49-f42aaacd65a6 None admin] PATH_INFO: `/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:14.766611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ee192-deb1-4cfd-af49-f42aaacd65a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5661ed33e05e4ac88b1efc9eb2b9f8cd)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:14.768066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ee192-deb1-4cfd-af49-f42aaacd65a6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:14.769683 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 573/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:14 2022] GET /identity/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:14.776481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:14.816597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:14.906617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e0bc-f8ec-49ad-8eb2-be0b7dbcdca6 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:14.949780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ad14d-4dcb-44e4-a361-3763cff28593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:14.950354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ad14d-4dcb-44e4-a361-3763cff28593 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:14.950624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ad14d-4dcb-44e4-a361-3763cff28593 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:14.950893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ad14d-4dcb-44e4-a361-3763cff28593 None admin] PATH_INFO: `/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd/users/60a9c49cd5234600b8ec0f3e83dd5852/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:14.952912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ad14d-4dcb-44e4-a361-3763cff28593 None admin] RBAC: Authorizing `identity:create_grant(project_id=5661ed33e05e4ac88b1efc9eb2b9f8cd, user_id=60a9c49cd5234600b8ec0f3e83dd5852, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:14.954418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ad14d-4dcb-44e4-a361-3763cff28593 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:14.970660 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 580/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:44:14 2022] PUT /identity/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd/users/60a9c49cd5234600b8ec0f3e83dd5852/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:44:17.078185 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7358a-ae68-46ec-97fb-ba6bc47f286e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:17.078851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7358a-ae68-46ec-97fb-ba6bc47f286e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:17.078851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7358a-ae68-46ec-97fb-ba6bc47f286e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:17.080851 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 574/1154] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:17.089823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:17.090232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:17.090370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:17.116095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:17.117284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:17.190136 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 581/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:17.458265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7358a-ae68-46ec-97fb-ba6bc47f286e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:17.593292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7358a-ae68-46ec-97fb-ba6bc47f286e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:17.628819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:17.629439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:17.629713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:17.629980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:17.641459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:44:17.646234 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:17.647410 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:44:17.648654 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 575/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:44:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:17.654109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:17.681440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:17.759266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b7867-538e-4392-a27d-f457887b27d5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:17.793760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:17.794341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:17.794605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:17.794868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:17.795558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:17.795847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:17.797230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:17.805264 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 582/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:44:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:17.811998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:17.816313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:17.916796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d8494-f06d-4452-aa38-d8f227d16dfc None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:17.951612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:17.952217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:17.952489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:17.952757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] PATH_INFO: `/v3/users/60a9c49cd5234600b8ec0f3e83dd5852` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:17.954064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] RBAC: Authorizing `identity:get_user(user_id=60a9c49cd5234600b8ec0f3e83dd5852)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:17.955525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:17.957075 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 576/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:17 2022] GET /identity/v3/users/60a9c49cd5234600b8ec0f3e83dd5852 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:17.966531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:18.068393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:18.083596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4477bf0-1bd8-43fb-8dc6-ecb1e5074a3d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:18.121361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:18.121971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:18.122202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:18.122470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:18.193985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:18.195458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:18.278580 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:18.294705 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 583/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:44:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:44:18.301570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:18.380769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:18.424950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8c6be-50cc-4931-aa4b-0a27ff665097 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:18.461462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a1497a-231c-4641-b9a3-51b82a8b687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:18.462080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1497a-231c-4641-b9a3-51b82a8b687b None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:18.462355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1497a-231c-4641-b9a3-51b82a8b687b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:18.462612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a1497a-231c-4641-b9a3-51b82a8b687b None admin] PATH_INFO: `/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:18.463874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1497a-231c-4641-b9a3-51b82a8b687b None admin] RBAC: Authorizing `identity:get_project(project_id=5661ed33e05e4ac88b1efc9eb2b9f8cd)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:18.465364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a1497a-231c-4641-b9a3-51b82a8b687b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:18.467004 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 577/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:18 2022] GET /identity/v3/projects/5661ed33e05e4ac88b1efc9eb2b9f8cd => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:18.474291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:18.586438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05ee22-7e98-4855-bdf2-b239b84b8b7c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:18.625575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:18.626160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:18.626433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:18.626700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:18.627453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=5661ed33e05e4ac88b1efc9eb2b9f8cd, user.id=60a9c49cd5234600b8ec0f3e83dd5852) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:18.627764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:18.629208 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2bfe4-4d37-434f-8585-d8a911bbea16 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:18.635672 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 584/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:44:18 2022] GET /identity/v3/role_assignments?user.id=60a9c49cd5234600b8ec0f3e83dd5852&scope.project.id=5661ed33e05e4ac88b1efc9eb2b9f8cd&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:20.764407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6870dce-1264-4461-99fe-46bf372d9f95 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:20.765021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6870dce-1264-4461-99fe-46bf372d9f95 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:20.765021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6870dce-1264-4461-99fe-46bf372d9f95 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:20.766333 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 578/1162] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:20.772778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:20.773474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:20.773634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:20.796537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:20.797835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:20.846023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:20.873635 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 585/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:21.139524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6870dce-1264-4461-99fe-46bf372d9f95 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:21.248035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6870dce-1264-4461-99fe-46bf372d9f95 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:21.284179 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:21.284765 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:21.285031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:21.285302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:21.295864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:21.353954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:21.355408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:21.454800 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 579/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:44:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:44:21.461478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:21.576079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245040b-f8c9-4a0a-8314-dda503121bcf None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:21.614366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:21.614982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:21.615288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:21.615553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:21.622301 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:21.623775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2bf27ba4-f3d3-46fa-80f5-2ed0b914a252 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:21.635288 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:21.636471 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:44:21.637347 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 586/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:44:21 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:21.642612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:21.687325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:21.747113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ed8a3-f038-479d-830c-9b2e9e9854b8 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:21.785864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:21.786488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:21.786703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:21.786969 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:21.787633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:21.787927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:21.789286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc3dfb-61c7-46e5-907d-e256a58b9b02 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:21.796896 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 580/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Sep 11 00:44:21 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:23.909724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252550-06d6-4518-908b-abd914b543ee None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:23.910638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252550-06d6-4518-908b-abd914b543ee None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:23.910638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74252550-06d6-4518-908b-abd914b543ee None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:23.912758 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 587/1167] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:23.921443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4374ec7-fc61-4894-b79f-67ec84895cfc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:23.921877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4374ec7-fc61-4894-b79f-67ec84895cfc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:23.921985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4374ec7-fc61-4894-b79f-67ec84895cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:23.941903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f4374ec7-fc61-4894-b79f-67ec84895cfc None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:23.943077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f4374ec7-fc61-4894-b79f-67ec84895cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:24.019879 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 581/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:24.291393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252550-06d6-4518-908b-abd914b543ee None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:24.338456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:24.414104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74252550-06d6-4518-908b-abd914b543ee None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:24.452577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da32ecb-2d78-456e-8f6d-e14e7d8e0e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:24.453184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da32ecb-2d78-456e-8f6d-e14e7d8e0e43 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:24.453453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da32ecb-2d78-456e-8f6d-e14e7d8e0e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:24.453743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da32ecb-2d78-456e-8f6d-e14e7d8e0e43 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:24.454416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da32ecb-2d78-456e-8f6d-e14e7d8e0e43 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:24.455802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da32ecb-2d78-456e-8f6d-e14e7d8e0e43 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:24.476062 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 588/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Sun Sep 11 00:44:24 2022] POST /identity/v3/domains => generated 267 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:44:26.627817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f72eb-6b87-41ef-9d8f-fa0b3572ed7f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:26.628715 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f72eb-6b87-41ef-9d8f-fa0b3572ed7f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:26.628715 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f72eb-6b87-41ef-9d8f-fa0b3572ed7f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:26.630860 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 582/1170] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:26.637138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:26.637592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:26.637685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:26.663834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:26.665011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:26.736837 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 589/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:27.008118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f72eb-6b87-41ef-9d8f-fa0b3572ed7f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:27.052471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:27.135844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f72eb-6b87-41ef-9d8f-fa0b3572ed7f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:27.172260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7eefd-ceca-4e4d-adda-867d4931eea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:27.172897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7eefd-ceca-4e4d-adda-867d4931eea7 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:27.173178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7eefd-ceca-4e4d-adda-867d4931eea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:27.173443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7eefd-ceca-4e4d-adda-867d4931eea7 None admin] PATH_INFO: `/v3/domains/b5206c7d96da4b0eb54c1dde21313577` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:27.184687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7eefd-ceca-4e4d-adda-867d4931eea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5206c7d96da4b0eb54c1dde21313577)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:27.186127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb7eefd-ceca-4e4d-adda-867d4931eea7 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:27.187732 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 583/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Sun Sep 11 00:44:27 2022] GET /identity/v3/domains/b5206c7d96da4b0eb54c1dde21313577 => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:27.194166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:27.231680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:27.310673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921af7d7-3aad-4ecb-936c-eaf3f0893e7b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:27.344827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d73a0-51d8-418a-95d9-e8d8e8840181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:27.345418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d73a0-51d8-418a-95d9-e8d8e8840181 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:27.345686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d73a0-51d8-418a-95d9-e8d8e8840181 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:27.345948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d73a0-51d8-418a-95d9-e8d8e8840181 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:27.346590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d73a0-51d8-418a-95d9-e8d8e8840181 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:27.348048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d73a0-51d8-418a-95d9-e8d8e8840181 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:27.376216 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 590/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:44:27 2022] POST /identity/v3/projects => generated 361 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:44:29.534543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37669a-2bd2-4929-a854-c5ac5c4500da None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:29.535432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37669a-2bd2-4929-a854-c5ac5c4500da None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:29.535432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37669a-2bd2-4929-a854-c5ac5c4500da None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:29.537505 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 584/1174] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:29.543064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:29.543520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:29.543609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:29.565040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:29.566217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:29.636935 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 591/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:29.903064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37669a-2bd2-4929-a854-c5ac5c4500da None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:30.012398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:30.022036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37669a-2bd2-4929-a854-c5ac5c4500da None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:30.058525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e79cd-4eec-4834-9459-54c47ec93033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:30.059143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e79cd-4eec-4834-9459-54c47ec93033 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:30.059444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e79cd-4eec-4834-9459-54c47ec93033 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:30.059705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e79cd-4eec-4834-9459-54c47ec93033 None admin] PATH_INFO: `/v3/domains/b5206c7d96da4b0eb54c1dde21313577` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:30.061017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e79cd-4eec-4834-9459-54c47ec93033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b5206c7d96da4b0eb54c1dde21313577)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:30.062441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385e79cd-4eec-4834-9459-54c47ec93033 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:30.064049 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 585/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Sun Sep 11 00:44:29 2022] GET /identity/v3/domains/b5206c7d96da4b0eb54c1dde21313577 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:30.070547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:30.088292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:30.173477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7fe5-0b33-47f4-9c14-d494d771cef4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:30.207664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25823f00-ddbe-4a45-8c86-a46bbaf152b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:30.208261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25823f00-ddbe-4a45-8c86-a46bbaf152b3 None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:30.208534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25823f00-ddbe-4a45-8c86-a46bbaf152b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:30.208806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25823f00-ddbe-4a45-8c86-a46bbaf152b3 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:30.209445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25823f00-ddbe-4a45-8c86-a46bbaf152b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:30.210878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25823f00-ddbe-4a45-8c86-a46bbaf152b3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:30.240100 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 592/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:44:30 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:44:32.431023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31398-9c60-4a00-a98e-aa1a70a12a0e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:32.431907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31398-9c60-4a00-a98e-aa1a70a12a0e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:32.431907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c31398-9c60-4a00-a98e-aa1a70a12a0e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:32.434439 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 586/1178] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:32.441659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:32.442104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:32.442204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:32.462752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:32.463990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:32.535242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:32.542542 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 593/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:32.816794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c31398-9c60-4a00-a98e-aa1a70a12a0e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:32.934447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c31398-9c60-4a00-a98e-aa1a70a12a0e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:32.970670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:32.971341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:32.971595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:32.971867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:32.983216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:44:32.988086 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:32.989169 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:44:32.990219 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 587/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:44:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:32.995746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:33.097930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372cac-e41e-4338-b01a-3131e5c666a3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:33.132574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:33.133165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:33.133436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:33.133702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:33.134361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:33.134661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:33.136055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:33.144351 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 594/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:44:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:44:33.153346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:33.179019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:33.259996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35368f9-22d4-4a94-a8e1-599e006852f1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:33.296147 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:33.296750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:33.297023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:33.297280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] PATH_INFO: `/v3/users/6c702a4be83448109bada9f8f550cabc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:33.311596 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] RBAC: Authorizing `identity:get_user(user_id=6c702a4be83448109bada9f8f550cabc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:33.313089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:33.314660 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 588/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:33 2022] GET /identity/v3/users/6c702a4be83448109bada9f8f550cabc => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:44:33.324067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:33.390593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:33.437166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6604b-d7c6-4528-8298-dcbf2b2cb6a0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:33.471602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:33.472197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:33.472451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:33.472712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:33.543937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:33.545381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:33.602668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:33.649163 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 595/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:44:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:44:33.656055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:33.746333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:33.766607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5906cc1d-ae64-4013-9e42-a7c95a525a63 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:33.802593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2494f0-0d71-4945-8cff-1b08a722e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:33.803220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2494f0-0d71-4945-8cff-1b08a722e122 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:33.803500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2494f0-0d71-4945-8cff-1b08a722e122 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:33.803767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2494f0-0d71-4945-8cff-1b08a722e122 None admin] PATH_INFO: `/v3/projects/4ac3d625283d4aba9d391a588e0f2926` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:33.804933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2494f0-0d71-4945-8cff-1b08a722e122 None admin] RBAC: Authorizing `identity:get_project(project_id=4ac3d625283d4aba9d391a588e0f2926)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:33.806364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2494f0-0d71-4945-8cff-1b08a722e122 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:33.808371 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 589/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:33 2022] GET /identity/v3/projects/4ac3d625283d4aba9d391a588e0f2926 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:33.818669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:33.927342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f460960-9dd8-4903-9f45-ffdc13984407 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:33.961832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:33.962414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:33.962680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:33.962942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:33.963685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=4ac3d625283d4aba9d391a588e0f2926, user.id=6c702a4be83448109bada9f8f550cabc) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:33.963983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:33.965400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974de44-eaa1-4ef1-8523-c7dfbe57ae8f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:33.970871 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 596/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:44:33 2022] GET /identity/v3/role_assignments?user.id=6c702a4be83448109bada9f8f550cabc&scope.project.id=4ac3d625283d4aba9d391a588e0f2926&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:36.050697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9af93-d78f-495b-a760-0f72593e75f2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:36.051305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9af93-d78f-495b-a760-0f72593e75f2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:36.051305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9af93-d78f-495b-a760-0f72593e75f2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:36.052579 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 590/1186] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:36.059237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:36.060115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:36.060115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:36.085345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:36.086530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:36.104474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:36.156746 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 597/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:36.420475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9af93-d78f-495b-a760-0f72593e75f2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:36.531962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9af93-d78f-495b-a760-0f72593e75f2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:36.569359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a6528-287c-4371-b354-9802555e848f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:36.569980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a6528-287c-4371-b354-9802555e848f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:36.570245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a6528-287c-4371-b354-9802555e848f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:36.570532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a6528-287c-4371-b354-9802555e848f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:36.581692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a6528-287c-4371-b354-9802555e848f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-b42a6528-287c-4371-b354-9802555e848f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:44:36.586418 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:36.587526 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:44:36.588514 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 591/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:44:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:36.596540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:36.710629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd89fa1-2530-476e-b53b-9bd2d3df54d1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:36.745120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:36.745708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:36.745978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:36.746247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:36.746900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:36.747219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:36.748595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:36.756743 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 598/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:44:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:36.765722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a6528-287c-4371-b354-9802555e848f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:36.778484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:36.872289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a6528-287c-4371-b354-9802555e848f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:36.908689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:36.909300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:36.909573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:36.909838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] PATH_INFO: `/v3/users/6c702a4be83448109bada9f8f550cabc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:36.911005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] RBAC: Authorizing `identity:get_user(user_id=6c702a4be83448109bada9f8f550cabc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:36.912476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:36.914054 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 592/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:36 2022] GET /identity/v3/users/6c702a4be83448109bada9f8f550cabc => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:36.920218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:36.939993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:37.022608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f214d-05b7-4bf9-ae5c-76402fa044bf None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:37.057793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fce8a-9ed4-489f-b3ab-5ed99c435b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:37.058420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fce8a-9ed4-489f-b3ab-5ed99c435b3d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:37.058687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fce8a-9ed4-489f-b3ab-5ed99c435b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:37.058934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fce8a-9ed4-489f-b3ab-5ed99c435b3d None admin] PATH_INFO: `/v3/projects/4ac3d625283d4aba9d391a588e0f2926` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:37.060344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fce8a-9ed4-489f-b3ab-5ed99c435b3d None admin] RBAC: Authorizing `identity:get_project(project_id=4ac3d625283d4aba9d391a588e0f2926)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:37.061780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fce8a-9ed4-489f-b3ab-5ed99c435b3d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:37.063520 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 599/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:36 2022] GET /identity/v3/projects/4ac3d625283d4aba9d391a588e0f2926 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:37.069795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:37.148301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:37.175235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1f3e5-60fa-4891-8fa3-ccf7160bffb1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:37.210867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f515649-5e35-4a0f-9890-4ce3caa7511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:37.211536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f515649-5e35-4a0f-9890-4ce3caa7511b None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:37.211803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f515649-5e35-4a0f-9890-4ce3caa7511b None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:37.212070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f515649-5e35-4a0f-9890-4ce3caa7511b None admin] PATH_INFO: `/v3/projects/4ac3d625283d4aba9d391a588e0f2926/users/6c702a4be83448109bada9f8f550cabc/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:37.214160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f515649-5e35-4a0f-9890-4ce3caa7511b None admin] RBAC: Authorizing `identity:create_grant(project_id=4ac3d625283d4aba9d391a588e0f2926, user_id=6c702a4be83448109bada9f8f550cabc, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:37.215617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f515649-5e35-4a0f-9890-4ce3caa7511b None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:37.230376 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 593/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:44:37 2022] PUT /identity/v3/projects/4ac3d625283d4aba9d391a588e0f2926/users/6c702a4be83448109bada9f8f550cabc/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:44:39.345862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2a5e0-c541-4e0d-bea9-8dbd6958ac70 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:39.346462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2a5e0-c541-4e0d-bea9-8dbd6958ac70 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:39.346462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2a5e0-c541-4e0d-bea9-8dbd6958ac70 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:39.347948 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 600/1193] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:44:39.354137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:39.354580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:39.354682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:39.381678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:44:39.382855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:44:39.452778 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 594/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:44:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:44:39.738720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2a5e0-c541-4e0d-bea9-8dbd6958ac70 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:39.832812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:39.850259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2a5e0-c541-4e0d-bea9-8dbd6958ac70 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:39.884079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:39.884695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:39.884968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:39.885236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:39.895674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:44:39.900460 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:44:39.901491 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:44:39.902528 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 601/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:44:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:44:39.908933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:39.976196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:40.029940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3bb62-94e5-426b-bd9a-397fce337d78 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:40.073180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:40.073780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:40.074050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:40.074311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:40.074973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:40.075297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:40.076664 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:40.085082 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 595/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:40.094575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:40.216199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa7764-0b69-45b6-babd-ea8e5f665137 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:40.253612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:40.254232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:40.254503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:40.254765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] PATH_INFO: `/v3/users/6c702a4be83448109bada9f8f550cabc` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:40.255945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] RBAC: Authorizing `identity:get_user(user_id=6c702a4be83448109bada9f8f550cabc)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:40.257387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:40.258983 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 602/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Sep 11 00:44:40 2022] GET /identity/v3/users/6c702a4be83448109bada9f8f550cabc => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:40.268123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:40.380690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440ef92-f272-4845-ad6f-b0611e685f21 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:40.407055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:40.421385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:40.422296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:40.422566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:40.422828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:40.493137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:40.494575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:40.594090 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 596/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:44:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:44:40.603172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:40.621075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:40.723147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e1719-e37a-41dd-a859-68da6e347436 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:40.758293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d8464-b993-4c79-a562-89e04cc6625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:40.758906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d8464-b993-4c79-a562-89e04cc6625e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:40.759202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d8464-b993-4c79-a562-89e04cc6625e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:40.759467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d8464-b993-4c79-a562-89e04cc6625e None admin] PATH_INFO: `/v3/projects/4ac3d625283d4aba9d391a588e0f2926` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:40.760723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d8464-b993-4c79-a562-89e04cc6625e None admin] RBAC: Authorizing `identity:get_project(project_id=4ac3d625283d4aba9d391a588e0f2926)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:40.762152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d8464-b993-4c79-a562-89e04cc6625e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:40.763817 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 603/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Sep 11 00:44:40 2022] GET /identity/v3/projects/4ac3d625283d4aba9d391a588e0f2926 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:44:40.770949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:44:40.803930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:40.881514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee3180-7081-423f-a6bd-89c80fbf6913 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:44:40.916649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:44:40.917243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:44:40.917515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:44:40.917786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:44:40.918507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=4ac3d625283d4aba9d391a588e0f2926, user.id=6c702a4be83448109bada9f8f550cabc) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:44:40.918813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:44:40.920269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a45b7-50e0-40fd-a767-eafcbfe68b60 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:44:40.926222 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 597/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:44:40 2022] GET /identity/v3/role_assignments?user.id=6c702a4be83448109bada9f8f550cabc&scope.project.id=4ac3d625283d4aba9d391a588e0f2926&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:44:50.753028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=60 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:44:50.923433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=54 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:36.627257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b02924-2109-4beb-aacc-b8b84881cf2b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:36.627257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b02924-2109-4beb-aacc-b8b84881cf2b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:36.627257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b02924-2109-4beb-aacc-b8b84881cf2b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:36.628557 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 604/1201] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:45:36.634525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:36.634999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:36.635088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:36.655044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:45:36.656582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:45:36.751012 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 598/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:45:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:45:37.021725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b02924-2109-4beb-aacc-b8b84881cf2b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:37.139922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b02924-2109-4beb-aacc-b8b84881cf2b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:37.184880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:37.185561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:37.185879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:37.186163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:37.198651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:37.200223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:37.215490 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:37.216602 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:37.218124 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:37.218124 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:37.218124 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:37.218124 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 605/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:37.223039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:37.347128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413230f-aed0-491a-bc68-24e73a2283d6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:37.381496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:37.388746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:37.389723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:37.389723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:37.389922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:37.390502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:37.390773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:37.392264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef9154-c8d2-4fec-81fc-5b39db2f9508 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:37.409363 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 599/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:37.411365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:37.479541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:37.542871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b6981-c7bf-41ed-be6f-0bababf2cc42 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:37.588960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e1eb-c7a3-4dd1-8c7e-5632e9f69faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:37.589750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e1eb-c7a3-4dd1-8c7e-5632e9f69faf None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:37.590026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e1eb-c7a3-4dd1-8c7e-5632e9f69faf None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:37.590303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e1eb-c7a3-4dd1-8c7e-5632e9f69faf None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:37.591243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e1eb-c7a3-4dd1-8c7e-5632e9f69faf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:37.592828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e1eb-c7a3-4dd1-8c7e-5632e9f69faf None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:37.628037 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 606/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Sep 11 00:45:37 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:45:39.946919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa951f-9e46-48f7-9fa0-92875b35c307 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:39.947513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa951f-9e46-48f7-9fa0-92875b35c307 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:39.947513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa951f-9e46-48f7-9fa0-92875b35c307 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:39.949224 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 600/1206] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:45:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:45:39.955446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:39.955984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:39.955984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:39.977205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:45:39.978817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:45:40.069277 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 607/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:45:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:45:40.367587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa951f-9e46-48f7-9fa0-92875b35c307 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:40.491127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa951f-9e46-48f7-9fa0-92875b35c307 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:40.531037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:40.531741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:40.532140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:40.532542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:40.544394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:45:40.550812 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:40.551831 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:40.552688 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 601/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:45:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:40.559235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:40.612230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:40.685233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2db414-a641-4b21-9b79-93ae0ad39d1b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:40.722469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:40.723829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:40.723829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:40.723829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:40.724542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:40.724844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:40.726485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:40.735449 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 608/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:45:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:40.742014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:40.803812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:40.863110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dd2f8-74e4-4b75-997c-92c279300265 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:40.900851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:40.901495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:40.901767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:40.902069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:40.913831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:40.915453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:40.931300 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:40.932341 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:40.933250 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:40.933250 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:40.933250 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:40.933250 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 602/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:40.938728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:41.059560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:41.062895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281289a-2554-4780-9b6b-6bb7255cc1fa None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:41.104457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:41.105254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:41.105527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:41.105789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:41.106467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:41.106760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:41.108184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:41.123731 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 609/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:45:41.129800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:41.249282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ed11b-e786-4622-9682-1037107dde15 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:41.250956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:41.289109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:41.289742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:41.290013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:41.290273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:41.298323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:41.299877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:45:41.311648 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:41.312693 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:41.313839 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 603/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:41.318710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:41.420302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70dcb30-2e81-4b7b-802d-beb245ec28ad None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:41.454939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:41.455602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:41.455871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:41.456133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:41.456877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:41.457170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:41.458583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:41.482449 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 610/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Sep 11 00:45:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:41.483907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:41.605362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4f08f-2c9b-4e6e-ba0f-26e15991f63f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:41.645332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:41.646103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:41.646470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:41.647307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:41.678708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:41.717847 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:41.719580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:41.844415 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 604/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:45:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:41.850582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:41.872254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:41.963745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e0fa2-258d-43dc-a6af-d5ebc78cc7c3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:42.004065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:42.004701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:42.004973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:42.005240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:42.017899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:42.019421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:42.034936 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:42.036394 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:42.037690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:42.037690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:42.037690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:42.037690 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 611/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:45:42.042241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:42.112807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:42.165986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad50bb-2382-4ec2-889f-4673e8932c7f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:42.205624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:42.206394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:42.206674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:42.206945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:42.207887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:42.210469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:42.210469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:42.222241 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 605/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:45:42.228322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:42.308967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:42.346421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c4ec3-84d0-40f4-8824-1c5c066f773c None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:42.392846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:42.393505 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:42.393775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:42.394027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:42.402198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:42.403684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:45:42.418390 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:42.419449 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:42.425677 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 612/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:42.426195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:42.542637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b48691b-cd98-4a4c-81e8-110ca254ef35 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:42.567293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:42.581420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:42.582122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:42.582609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:42.582838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:42.583652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:42.584010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:42.585740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9e971-4619-41b6-b4c4-e265918f9338 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:42.599814 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 606/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:45:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:42.606353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:42.712965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab83faf-8118-4220-abe4-c559cdb985ec None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:42.741475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:42.747714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:42.748286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:42.748550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:42.748811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:42.749528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=00e3a0e394ba43f3a2362c3ac69a2fa6) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:42.749876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:42.751365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc9a16-fdf1-4626-bec9-3f0a8bbf1f83 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:42.756657 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 613/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:45:42 2022] GET /identity/v3/role_assignments?user.id=00e3a0e394ba43f3a2362c3ac69a2fa6&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:45:44.970844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f312d-92c2-4eac-92c9-ca3ca60107cd None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:44.971449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f312d-92c2-4eac-92c9-ca3ca60107cd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:44.971449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f312d-92c2-4eac-92c9-ca3ca60107cd None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:44.973008 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 607/1220] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:45:44.978752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:44.979259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:44.979259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:44.997817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:45:44.998992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:45:45.065558 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 614/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:45:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:45:45.352510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f312d-92c2-4eac-92c9-ca3ca60107cd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:45.473065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f312d-92c2-4eac-92c9-ca3ca60107cd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:45.510228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:45.511707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:45.512036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:45.512508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:45.525610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:45:45.530325 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:45.531348 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:45.532269 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 608/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:45:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:45.537974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:45.655956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26320c8-71f3-47da-880f-78c6244c2494 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:45.681344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:45.693479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:45.694109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:45.694379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:45.694639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:45.695333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] RBAC: Adding query filter params (name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:45.695636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:45.696970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:45.705106 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 615/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:45.711208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:45.759767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:45.822435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2010d26-5069-4033-88ef-5e4237dfdfcd None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:45.858950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd74630-926e-4493-8354-779175c51752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:45.859601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74630-926e-4493-8354-779175c51752 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:45.859873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74630-926e-4493-8354-779175c51752 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:45.860174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd74630-926e-4493-8354-779175c51752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:45.870975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd74630-926e-4493-8354-779175c51752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:45.872466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd74630-926e-4493-8354-779175c51752 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-7dd74630-926e-4493-8354-779175c51752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:45.887897 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:45.889001 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:45.889929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:45.889929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:45.889929 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:45.889929 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 609/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:45:45.895274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:45.998063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb5e8e3-7d16-4799-b8f3-ff55071f412d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:46.036817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:46.037440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:46.037759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:46.038036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:46.038775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:46.039105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:46.040552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:46.053058 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 616/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:46.059838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd74630-926e-4493-8354-779175c51752 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:46.174819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:46.187441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd74630-926e-4493-8354-779175c51752 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:46.227889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:46.228504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:46.228771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:46.229052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:46.236883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:46.238322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:45:46.249527 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:46.250731 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:46.251749 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 610/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:46.256568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:46.318082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:46.373828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ea175-bc71-4d90-9a60-cf661fbb4ae0 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:46.409415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:46.410021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:46.410291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:46.410562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:46.411313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:46.411611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:46.413007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:46.430164 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 617/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Sep 11 00:45:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:46.436432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:46.555303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0301c88-8a98-4e53-b1cd-69d88952e66e None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:46.593621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:46.594347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:46.594695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:46.594995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:46.606474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:46.607961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:46.624125 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:46.625264 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:46.626996 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:46.626996 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:46.626996 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:46.626996 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 611/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:46.631321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:46.737780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b533efd-382d-439d-b73e-edf7b1eac9f3 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:46.740290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:46.773468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:46.774075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:46.774347 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:46.774599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:46.775271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:46.775562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:46.776986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:46.788214 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 618/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:46.794148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:46.807620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:46.903401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e21fb-6b51-4169-bd0b-349681a5adbe None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:46.942103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:46.942745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:46.943000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:46.943308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:46.950780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:46.952354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:45:46.963928 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:46.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:46.965883 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 612/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:45:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:46.970955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:47.083697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe411d6-35a4-4f4a-9b9c-b78e94c1ec98 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:47.120265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:47.120873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:47.121141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:47.121403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:47.122112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:47.122406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:47.123903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1b008-ec99-46a8-8aea-bf2c24b97393 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:47.136444 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 619/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:45:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:47.142846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:47.216353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:47.254172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f297-1663-4b93-8742-902b63de15ad None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:47.295637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d5350-25e8-45b6-9c2f-98795097e9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:47.296508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d5350-25e8-45b6-9c2f-98795097e9aa None admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:47.296890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d5350-25e8-45b6-9c2f-98795097e9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:47.297253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d5350-25e8-45b6-9c2f-98795097e9aa None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/00e3a0e394ba43f3a2362c3ac69a2fa6/roles/757f5ddcfa4b4c08b12557285e677dfb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:47.315268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d5350-25e8-45b6-9c2f-98795097e9aa None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=00e3a0e394ba43f3a2362c3ac69a2fa6, role_id=757f5ddcfa4b4c08b12557285e677dfb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:47.316728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d5350-25e8-45b6-9c2f-98795097e9aa None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:47.330475 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 613/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:45:47 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/00e3a0e394ba43f3a2362c3ac69a2fa6/roles/757f5ddcfa4b4c08b12557285e677dfb => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:45:49.590747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9908c-4d1a-4923-b8b8-dce970425432 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:49.591378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9908c-4d1a-4923-b8b8-dce970425432 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:49.591378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9908c-4d1a-4923-b8b8-dce970425432 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:49.593052 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 620/1233] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:45:49.599871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:49.600475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:49.600475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:49.619732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:45:49.620940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:45:49.692904 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 614/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:45:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 11 00:45:49.960212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9908c-4d1a-4923-b8b8-dce970425432 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:49.986054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:50.070237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9908c-4d1a-4923-b8b8-dce970425432 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:50.106746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:50.107416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:50.107733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:50.107948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:50.118380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:45:50.123078 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:50.124223 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:50.125619 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 621/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Sep 11 00:45:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:45:50.131169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:50.185135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:50.242816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a464c3a-b412-427a-b9b3-1c839ad9f649 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:50.281462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:50.282421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:50.282696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:50.282967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:50.283704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:50.284000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:50.285388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:50.294895 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 615/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Sep 11 00:45:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:50.300979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:50.390625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:50.407914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902bf3f-56da-4256-b4d3-37621cc39406 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:50.444815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:50.445582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:50.445854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:50.446118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:50.456707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:50.458152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:50.472566 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:50.473559 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:50.474514 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:50.474514 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:50.474514 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:50.474514 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 622/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:45:50.479922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:50.587235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb07e8-f51a-431e-826f-f6d4f5ac97d0 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:50.597977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:50.632512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:50.633324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:50.633428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:50.633688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:50.634347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:50.634648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:50.636121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:50.649596 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 616/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:50.655802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:50.760925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4f83-5508-4509-a88f-07e4442ec2cc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:50.801681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:50.802315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:50.802590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:50.802857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:50.810380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:50.811757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:50.813679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:45:50.827116 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:50.828257 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:50.829301 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 623/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:50.835109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:50.950967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5bd61-cf59-43fc-b357-56785cfed4b9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:50.985057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:50.985768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:50.985998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:50.986286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:50.987052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:50.987342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:50.988785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:51.006491 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 617/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Sep 11 00:45:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:51.012499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:51.121203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb6419-8916-4a1d-8543-d33a50405e58 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:51.156687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:51.157513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:51.157778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:51.158096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:51.222001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:51.228844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:51.230323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:51.344125 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 624/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:45:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 00:45:51.350645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:51.380518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:51.461344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce74dcfe-d78e-40f2-9441-8ec3a8a4ab5d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:51.499617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:51.500179 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:51.500592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:51.500881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:51.512661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:51.514195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:51.531338 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:51.532663 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:51.533813 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:51.533813 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:51.533813 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:51.533813 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 618/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:51.538827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:51.646770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:51.656006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e9849-48b3-4bce-80a8-c318680c6a44 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:51.696977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:51.697798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:51.698164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:51.698490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:51.699292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:51.699718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:51.701331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:51.714659 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 625/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:51.720238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:51.808578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:51.832917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f954e4-4f9a-4b7e-99b3-babba118d148 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:51.869091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:51.869785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:51.870027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:51.870249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:51.877727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:51.879700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:45:51.892937 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:51.894110 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:51.895168 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 619/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:51.900525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:52.002103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b674ee7-c557-4e59-9860-ff6cc4001014 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:52.041786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:52.042521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:52.042833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:52.043156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:52.043942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:52.044273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:52.045885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda497ad-16c2-4e3c-8faf-0cce9a145e3d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:52.059198 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 626/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:45:52.065431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:52.178108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d671bcf-ac6c-4a6c-bdd2-c20fb9b697d9 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:52.215918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:52.216488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:52.216760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:52.217020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:52.217751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] RBAC: Adding query filter params (role.id=757f5ddcfa4b4c08b12557285e677dfb, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=00e3a0e394ba43f3a2362c3ac69a2fa6) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:52.218056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:52.219527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19803474-4570-4976-9b87-44fbccce6752 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:52.225523 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 620/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:45:52 2022] GET /identity/v3/role_assignments?user.id=00e3a0e394ba43f3a2362c3ac69a2fa6&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=757f5ddcfa4b4c08b12557285e677dfb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:54.551683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825ec49-aa04-4873-8518-51cb8022046f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:54.552210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825ec49-aa04-4873-8518-51cb8022046f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:54.552210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825ec49-aa04-4873-8518-51cb8022046f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:54.553712 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 627/1247] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:45:54.559704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:54.560186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:54.560309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:54.566796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:54.579580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:45:54.580760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:45:54.660067 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 621/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:45:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:45:54.938159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825ec49-aa04-4873-8518-51cb8022046f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:54.963253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:55.058125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825ec49-aa04-4873-8518-51cb8022046f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:55.097941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:55.098573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:55.098816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:55.099077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:55.110470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:45:55.115169 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:55.116204 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:55.117157 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 628/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:45:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:45:55.122608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:55.232479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200fa740-ae82-4fef-8615-ba3124bc93f4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:55.267433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:55.273435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:55.274321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:55.274793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:55.275085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:55.275899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:55.276214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:55.277586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:55.288021 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 622/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:55.294927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:55.383964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:55.413749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c90d6d-f8e3-40ce-a100-c8573d254403 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:55.457214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:55.457840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:55.458141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:55.458369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:55.471843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:55.473576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:55.490083 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:55.491561 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:55.492651 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:55.492651 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:55.492651 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:55.492651 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 629/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:55.497868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:55.616982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ff596-dcdc-4bc3-b81b-ba36f3fbd2a1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:55.656164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:55.656769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:55.657037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:55.657315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:55.657936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:55.658231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:55.659627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:55.676102 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 623/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:45:55.678245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:55.779143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f2338-b2bf-43af-acba-7e4b9c9d73b1 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:55.806599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:55.814746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:55.815397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:55.815664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:55.815924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:55.823917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:55.825800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:45:55.840309 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:55.841360 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:45:55.842371 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:55.844352 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 630/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:55.851064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:55.931394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:55.978852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e3492-3720-4daa-a19a-e41db3675539 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:56.020998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:56.022062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:56.022504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:56.022966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:56.024314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:56.024880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:56.027582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:56.047327 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 624/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Sep 11 00:45:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:56.053775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:56.169198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018c78c-b879-45ce-87d4-b195674ec681 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:56.208260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:56.208869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:56.209140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:56.209402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:56.272447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:56.285097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:56.286557 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:56.397192 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 631/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:45:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 11 00:45:56.404054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:56.523287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee93ae2-faed-406d-b1d8-521c9acb31c6 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:56.540181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:56.571500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:56.572134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:56.572407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:56.572673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:56.584061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:56.585724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:45:56.602805 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:56.604340 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:45:56.605354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:45:56.605354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:45:56.605354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:56.605354 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 625/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:56.610400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:56.706176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:56.719900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1251bf-ab33-4c99-ab92-5de558261ebe None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:56.754869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:56.755482 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:56.755755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:56.756018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:56.756664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:56.756963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:56.758368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:56.770054 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 632/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:56.777471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:56.889511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c2c00-b390-48db-86ef-9fdaf1ecb312 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:56.925670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:56.926325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:56.926594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:56.926852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:56.934846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:56.936587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:45:56.949016 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:56.950140 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:45:56.951367 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 626/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:56.956607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:57.059791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a46a0-88cf-4a1f-bf7f-255a70d3da7b None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:57.096348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:57.096966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:57.097245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:57.097504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:57.098205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:57.098503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:57.099984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f58c54-bd77-4893-a233-cdc806dc0f1b None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:57.113215 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 633/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:57.120042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:57.151235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:57.232873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368a116-bbb5-4705-825a-a0bc7ac548af None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:57.268836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:57.269455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:57.269729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:57.269995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:57.270725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=00e3a0e394ba43f3a2362c3ac69a2fa6) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:45:57.271029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:45:57.272482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00acb25-6706-4b06-9e6a-458234cfcdf1 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:45:57.278059 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 627/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:45:57 2022] GET /identity/v3/role_assignments?user.id=00e3a0e394ba43f3a2362c3ac69a2fa6&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=77132358e0b64de4a63750b81e359340 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:45:59.413457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf53291-1f8f-4cf5-8044-c562b405ff58 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:59.414011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf53291-1f8f-4cf5-8044-c562b405ff58 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:59.414011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf53291-1f8f-4cf5-8044-c562b405ff58 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:59.415318 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 634/1261] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:45:59.421644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:59.422479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:59.422633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:59.454221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:45:59.455448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:45:59.530229 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 628/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:45:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:45:59.806223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf53291-1f8f-4cf5-8044-c562b405ff58 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:45:59.818541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:45:59.916299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf53291-1f8f-4cf5-8044-c562b405ff58 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:45:59.951526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:45:59.952134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:45:59.952400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:45:59.952663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:45:59.962946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:45:59.967648 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:45:59.968751 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:45:59.970475 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 635/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:45:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:45:59.978837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:00.006304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:00.089671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e765d-b9be-4d0f-ae0c-712c890f199f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:00.124179 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:00.124781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:00.125053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:00.125416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:00.126208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:00.126504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:00.127904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:00.137023 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 629/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:45:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:00.143042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:00.217199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:00.254385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d28cf-2604-403c-961e-01ace1b18697 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:00.293147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:00.293761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:00.294048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:00.294303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:00.304706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:00.306499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:46:00.321985 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:00.323050 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:46:00.324026 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:46:00.324026 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:00.324026 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:46:00.324026 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 636/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:00.329151 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:00.411027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:00.434758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4a8cc-e533-4520-97b2-67e73b2381bb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:00.470327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:00.470917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:00.471218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:00.471478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:00.472118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:00.472409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:00.473780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:00.486575 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 630/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:00.492633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:00.597508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418caae-2e45-4ee7-8655-07b38dbc332a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:00.626400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:00.643389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:00.644101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:00.644369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:00.644632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:00.653290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:00.654735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:46:00.670581 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:00.672183 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:46:00.673291 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 637/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:46:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:00.679069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:00.784581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f377-4d7d-492d-bfe7-d24b33a3879d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:00.820194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:00.820787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:00.821050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:00.821290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:00.822008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:00.822433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:00.823773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:00.835502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:00.847664 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 631/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Sep 11 00:46:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:00.853814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:00.957553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c49cc-3abf-49c4-9d88-785e1a0af7dc None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:00.993567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:00.994189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:00.994455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:00.994721 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:01.007370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:01.008833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:46:01.025389 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:01.026456 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:46:01.027573 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:46:01.027573 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:01.027573 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:46:01.027573 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 638/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:01.034634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:01.141631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23abb5-465d-4256-9238-2b12a60ed98d None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:01.178724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:01.179360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:01.179631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:01.179899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:01.180905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:01.181202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:01.182569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:01.193744 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 632/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:01.202452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:01.235125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:01.308875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078f5ed-39c2-4d9a-8c7d-7e6f7e4d1e1f None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:01.343452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:01.344149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:01.344421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:01.344684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:01.351600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:01.353048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:46:01.364770 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:01.365816 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:46:01.367059 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 639/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:46:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 11 00:46:01.375346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:01.435887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:01.485358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea21aea-9c28-4d3e-b6f7-9917aa8b71fb None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:01.530065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:01.530922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:01.531431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:01.531788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:01.532485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:01.532790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:01.534223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbde8b-b852-46e6-b97b-54919b1831fc None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:01.546833 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 633/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:46:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:01.553175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:01.653362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:01.663918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6583-4c5a-4719-9df3-0e717d1b448d None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:01.700875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258e807-afe6-47c1-a51f-2d3bc3219ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:01.701512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258e807-afe6-47c1-a51f-2d3bc3219ee4 None admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:01.701785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258e807-afe6-47c1-a51f-2d3bc3219ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:01.702050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258e807-afe6-47c1-a51f-2d3bc3219ee4 None admin] PATH_INFO: `/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/00e3a0e394ba43f3a2362c3ac69a2fa6/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:01.704047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258e807-afe6-47c1-a51f-2d3bc3219ee4 None admin] RBAC: Authorizing `identity:create_grant(project_id=be8a77b9c6d04dfb826b0a857fcb6087, user_id=00e3a0e394ba43f3a2362c3ac69a2fa6, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:01.705469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258e807-afe6-47c1-a51f-2d3bc3219ee4 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:01.720686 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 640/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Sep 11 00:46:01 2022] PUT /identity/v3/projects/be8a77b9c6d04dfb826b0a857fcb6087/users/00e3a0e394ba43f3a2362c3ac69a2fa6/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:46:03.834316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2930-1bd6-4f58-8491-51fc1f62f79c None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:03.834976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2930-1bd6-4f58-8491-51fc1f62f79c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:03.834976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b2930-1bd6-4f58-8491-51fc1f62f79c None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:03.836243 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 634/1274] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:46:03.842528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:03.843282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:03.843460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:03.863866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:46:03.865075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:46:03.933194 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 641/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:46:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:46:04.197127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b2930-1bd6-4f58-8491-51fc1f62f79c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:04.305657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b2930-1bd6-4f58-8491-51fc1f62f79c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:04.323080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:04.343438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:04.344158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:04.344431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:04.344693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:04.356365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 11 00:46:04.361130 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:04.362290 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:04.363541 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 635/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Sep 11 00:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:04.368331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:04.448909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:04.468304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0e5ed5-75de-4d3f-9f2c-a04a9e6a408f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:04.502576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:04.503167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:04.503472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:04.503737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:04.504398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:04.504694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:04.506058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:04.514295 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 642/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Sep 11 00:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:46:04.521189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:04.635047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04e8ab-4201-4560-8acb-bb42c151b7f1 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:04.689845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:04.690470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:04.690738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:04.690998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:04.701676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:04.703048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:46:04.717757 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:04.723554 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:46:04.724589 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:46:04.724589 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:04.724589 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:04.724589 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 636/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:04.724848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:04.834235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d822603-8c8e-4540-bb72-cca11ab2ae90 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:04.870465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:04.871083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:04.871353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:04.871622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:04.872261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:04.872560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:04.873919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:04.885691 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 643/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:04.892492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:04.936021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:05.018083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640b074-1193-41f7-9a6a-9deb8fbb5fea None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:05.068795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:05.069425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:05.069699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:05.069969 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:05.077527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:05.078986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 11 00:46:05.089975 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:05.095310 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:05.096405 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 637/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:46:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:05.097319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:05.102035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:05.216684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0736b-06c7-498e-b4cc-019ed83c2af2 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:05.251610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:05.252209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:05.252483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:05.252752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:05.253474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:05.253779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:05.255225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:05.272897 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 644/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Sep 11 00:46:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:05.281794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:05.375577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:05.386062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe456-2e13-4092-83c6-fcf4a3bf08d2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:05.421748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:05.422376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:05.422651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:05.422916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:05.492323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:05.493745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:05.589843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:05.600705 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 638/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Sep 11 00:46:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 11 00:46:05.606599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:05.668862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:05.719572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d8fd-807e-4f97-81fb-bcb3ea65d161 None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:05.754517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:05.755126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:05.755434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:05.755709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:05.765693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:05.767128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 11 00:46:05.781431 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:05.782521 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 11 00:46:05.784180 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 11 00:46:05.784180 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 11 00:46:05.784180 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:46:05.784180 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 645/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:05.791885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:05.907409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7949e008-5e80-47cd-9e07-2309e57dae46 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:05.951648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:05.952515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:05.952867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:05.953243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:05.954200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:05.954616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:05.956766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:05.973138 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 639/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Sep 11 00:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:05.982034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:06.089528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91ed17-67b8-403a-821e-0d2bd365b06a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:06.104130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:06.126989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:06.127637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:06.127908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:06.128178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:06.134748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:06.136220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 11 00:46:06.147690 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:06.148777 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: ERROR keystone.server.flask.application Sep 11 00:46:06.150006 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 646/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Sep 11 00:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:06.154882 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:06.206173 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:06.256101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbaad2f-f13e-4147-b2a0-8a579c3ab326 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:06.291577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:06.292163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:06.292432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:06.292688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:06.293369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:06.293663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:06.295073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68515df1-e083-44a0-9816-8e451dee051c None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:06.307460 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 640/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Sep 11 00:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:06.313995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:06.418597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caab6a-3f5f-42f2-ac26-9b36ebf0b10a None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:06.453489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:06.454068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:06.454342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:06.454602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:06.455341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] RBAC: Adding query filter params (role.id=77132358e0b64de4a63750b81e359340, scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087, user.id=00e3a0e394ba43f3a2362c3ac69a2fa6) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:06.455646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:06.457039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1fa45-7859-4660-b75d-faa246de0a15 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:06.462882 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 647/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Sep 11 00:46:06 2022] GET /identity/v3/role_assignments?user.id=00e3a0e394ba43f3a2362c3ac69a2fa6&scope.project.id=be8a77b9c6d04dfb826b0a857fcb6087&role.id=77132358e0b64de4a63750b81e359340 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:08.584283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e44ba-ed91-425d-9a71-9e0a35814b12 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:08.585251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e44ba-ed91-425d-9a71-9e0a35814b12 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:08.585251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e44ba-ed91-425d-9a71-9e0a35814b12 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:08.587753 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 641/1288] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:46:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:46:08.594326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:08.594774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:08.594999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:08.620661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:46:08.621825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:46:08.660616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:08.707867 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 648/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:46:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:46:08.968828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e44ba-ed91-425d-9a71-9e0a35814b12 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:09.088610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e44ba-ed91-425d-9a71-9e0a35814b12 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:09.099314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:09.127663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:09.128276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:09.128545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:09.128808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:09.129382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:09.130749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:09.139714 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:09.140878 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:09.142191 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 642/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:46:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:09.148618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:09.258042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd068aae-a4a2-455d-b246-70d7f7aaa14f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:09.297590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:09.298195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:09.298472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:09.298743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:09.299403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:09.299705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:09.301068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc2b49-df4f-46de-936b-099e98c4b51e None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:09.306423 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 649/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Sep 11 00:46:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:09.313073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:09.422873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9205e66-8b0b-409d-b23b-bd59d059d03f None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:09.458482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:09.459055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:09.459355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:09.459613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] PATH_INFO: `/v3/services` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:09.460232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:09.460527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:09.461878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54efe-934c-4b19-8ed2-1ae6159a70a9 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:09.468377 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 643/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Sep 11 00:46:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:11.586810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3bfb9-31b8-40ab-a53d-cded1d17f699 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:11.587480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3bfb9-31b8-40ab-a53d-cded1d17f699 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:11.587480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3bfb9-31b8-40ab-a53d-cded1d17f699 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:11.588877 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 650/1293] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:46:11.595669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc837154-7342-4bbb-a5df-e52f6b977a28 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:11.596334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc837154-7342-4bbb-a5df-e52f6b977a28 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:11.596557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc837154-7342-4bbb-a5df-e52f6b977a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:11.622778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fc837154-7342-4bbb-a5df-e52f6b977a28 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:46:11.624137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fc837154-7342-4bbb-a5df-e52f6b977a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:46:11.658295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:11.713184 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 644/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:46:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:46:11.983742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3bfb9-31b8-40ab-a53d-cded1d17f699 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:12.017858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:12.089499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3bfb9-31b8-40ab-a53d-cded1d17f699 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:12.128302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85c55a-4e55-4703-8634-cf64f163f36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:12.128905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85c55a-4e55-4703-8634-cf64f163f36f None admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:12.129174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85c55a-4e55-4703-8634-cf64f163f36f None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:12.129436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85c55a-4e55-4703-8634-cf64f163f36f None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:12.130083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85c55a-4e55-4703-8634-cf64f163f36f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:12.131474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85c55a-4e55-4703-8634-cf64f163f36f None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:12.149385 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 651/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Sep 11 00:46:11 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:46:14.280310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d793c8d-715d-4ded-a5c8-3e756d714467 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:14.281200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d793c8d-715d-4ded-a5c8-3e756d714467 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:14.281200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d793c8d-715d-4ded-a5c8-3e756d714467 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:14.283100 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 645/1296] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:46:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:46:14.289529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:14.289965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:14.290083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:14.317961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:46:14.319143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:46:14.401612 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 652/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:46:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:46:14.669779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d793c8d-715d-4ded-a5c8-3e756d714467 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:14.789684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d793c8d-715d-4ded-a5c8-3e756d714467 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:14.847317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:14.847948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:14.848217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:14.848479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:14.849061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:14.850493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:14.862354 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:14.863635 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:14.865638 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 646/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:46:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:14.873623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:14.896816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:14.996571 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82d05c-d69b-4f1d-9f5c-da598a6989b9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:15.034895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:15.035527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:15.035799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:15.036065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:15.036691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:15.036987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:15.038356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:15.043772 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 653/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Sep 11 00:46:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:15.050131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:15.082894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:15.165586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02acda-ff8a-4440-97ac-3af2de70fd95 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:15.205410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:15.206009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:15.206278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:15.206544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:15.207233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] RBAC: Adding query filter params (interface=public, service_id=6575e998a6c44d298561da5d0d289272, region_id=RegionOne) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:15.207555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:15.208916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3078ff4-2461-4c0e-b415-1f5381e6e1c4 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:15.214940 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 647/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Sep 11 00:46:15 2022] GET /identity/v3/endpoints?service_id=6575e998a6c44d298561da5d0d289272&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:15.223428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:15.342438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:15.362855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf7292-1fe2-47e7-8c10-bf68f92828ce None admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:15.398233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c98fc-0a73-443c-af6b-74a4269e9abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:15.398826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c98fc-0a73-443c-af6b-74a4269e9abd None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:15.399099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c98fc-0a73-443c-af6b-74a4269e9abd None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:15.399396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c98fc-0a73-443c-af6b-74a4269e9abd None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:15.399997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c98fc-0a73-443c-af6b-74a4269e9abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:15.401357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853c98fc-0a73-443c-af6b-74a4269e9abd None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:15.408200 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 654/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Sep 11 00:46:15 2022] GET /identity/v3/services => generated 2191 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:46:17.517356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175953-eaf8-46cc-9e3a-8a5352b2bbd2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:17.518177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175953-eaf8-46cc-9e3a-8a5352b2bbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:17.518177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a175953-eaf8-46cc-9e3a-8a5352b2bbd2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:17.520239 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 648/1302] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:46:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:46:17.525622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:17.526073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:17.526180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:17.546249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:46:17.547465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:46:17.614587 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 655/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:46:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 11 00:46:17.877174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175953-eaf8-46cc-9e3a-8a5352b2bbd2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:17.996204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:18.005028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a175953-eaf8-46cc-9e3a-8a5352b2bbd2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:18.045315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:18.045934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:18.046198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:18.046463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:18.047035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:18.048435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 11 00:46:18.057893 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 11 00:46:18.059059 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.get_or_create( Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application with Lock( Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self._enter() Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application return self.creator() Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application created_value = creator( Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: ERROR keystone.server.flask.application Sep 11 00:46:18.060337 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 649/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Sep 11 00:46:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 11 00:46:18.065318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:18.144235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:18.170891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af821dd6-aace-4957-a698-38c81bfa3dc7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:18.207053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:18.207690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:18.207963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:18.208227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] PATH_INFO: `/v3/services` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:18.208849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:46:18.209148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:18.210486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fefc3-a7fa-4cd8-ae95-0b6cd3fd3f57 None admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:18.215580 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 656/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Sep 11 00:46:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:46:18.222026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:46:18.333166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe337aa0-a200-4caf-9148-c5d02e5f79b2 None admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:46:18.368634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73f9f0-6577-452d-8895-42c4d529c268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:46:18.369219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73f9f0-6577-452d-8895-42c4d529c268 None admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:46:18.369482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73f9f0-6577-452d-8895-42c4d529c268 None admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:46:18.369744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73f9f0-6577-452d-8895-42c4d529c268 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:46:18.370396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73f9f0-6577-452d-8895-42c4d529c268 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:46:18.371788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73f9f0-6577-452d-8895-42c4d529c268 None admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:46:18.393602 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 650/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:46:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:46:28.212499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=30 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:46:28.389392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:22.881437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f6cc3-5bca-4541-bc77-fcfae55074cb None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:22.881924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f6cc3-5bca-4541-bc77-fcfae55074cb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:22.882006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668f6cc3-5bca-4541-bc77-fcfae55074cb None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:22.883447 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 657/1307] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:47:22.889641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcaad2-a3d6-4df9-8ca0-ec58551e0aa7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:22.890131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcaad2-a3d6-4df9-8ca0-ec58551e0aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:22.890419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcaad2-a3d6-4df9-8ca0-ec58551e0aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:23.002744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-26bcaad2-a3d6-4df9-8ca0-ec58551e0aa7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:23.004524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-26bcaad2-a3d6-4df9-8ca0-ec58551e0aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:23.144834 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 651/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:23.430502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f5468-0589-4814-b986-713fd65229b9 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:23.430759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f5468-0589-4814-b986-713fd65229b9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:23.431131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f5468-0589-4814-b986-713fd65229b9 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:23.432562 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 658/1309] 10.222.0.23 () {54 vars in 965 bytes} [Sun Sep 11 00:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:23.438872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa474d-5006-48f9-88b9-617a898ee99f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:23.439326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa474d-5006-48f9-88b9-617a898ee99f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:23.439663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa474d-5006-48f9-88b9-617a898ee99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:23.520376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3baa474d-5006-48f9-88b9-617a898ee99f None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:23.521691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3baa474d-5006-48f9-88b9-617a898ee99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:23.543773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:23.610747 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 652/1310] 10.222.0.23 () {58 vars in 1062 bytes} [Sun Sep 11 00:47:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:23.620534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f5468-0589-4814-b986-713fd65229b9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:23.622482 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ce8f5468-0589-4814-b986-713fd65229b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:23.752783 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f5468-0589-4814-b986-713fd65229b9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:23.837416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:23.838420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:23.838920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:23.839435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:23.841487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:23.909756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:23.978285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:23.979678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff283d6-35e7-4989-9fcc-ff8aad6091c0 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:23.999654 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 659/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:26.417531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14831afd-9028-43c8-a311-f36388a930ff None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:26.418092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14831afd-9028-43c8-a311-f36388a930ff None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:26.418092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14831afd-9028-43c8-a311-f36388a930ff None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:26.419456 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 653/1312] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:26.425061 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21179a-1b91-4e75-9945-6d6fe00aa9f3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:26.425555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21179a-1b91-4e75-9945-6d6fe00aa9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:26.425660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21179a-1b91-4e75-9945-6d6fe00aa9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:26.509079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8e21179a-1b91-4e75-9945-6d6fe00aa9f3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:26.510287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8e21179a-1b91-4e75-9945-6d6fe00aa9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:26.612434 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 660/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:47:26.907737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc349f-e6a1-4953-8be3-756acb525cb3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:26.907998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc349f-e6a1-4953-8be3-756acb525cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:26.908305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc349f-e6a1-4953-8be3-756acb525cb3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:26.909632 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 654/1314] 10.222.0.23 () {54 vars in 965 bytes} [Sun Sep 11 00:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:26.915848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214fdcf-33b3-4ea0-9ecf-3fcdf8961ba7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:26.916225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214fdcf-33b3-4ea0-9ecf-3fcdf8961ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:26.916561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214fdcf-33b3-4ea0-9ecf-3fcdf8961ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:26.930218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:27.001768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a214fdcf-33b3-4ea0-9ecf-3fcdf8961ba7 None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:27.003333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a214fdcf-33b3-4ea0-9ecf-3fcdf8961ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:27.083743 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 661/1315] 10.222.0.23 () {58 vars in 1062 bytes} [Sun Sep 11 00:47:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:27.093362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc349f-e6a1-4953-8be3-756acb525cb3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:27.095025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9adc349f-e6a1-4953-8be3-756acb525cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:27.220010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc349f-e6a1-4953-8be3-756acb525cb3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:27.301930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:27.304850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:27.305479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:27.305794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:27.306050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:27.308209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:27.423309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:27.424672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356be52b-62bd-49f3-bcce-14ae3bd92c7e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:27.443354 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 655/1316] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:30.147302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ba316-3d04-4f1b-b0f2-6eb73719e774 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:30.147893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ba316-3d04-4f1b-b0f2-6eb73719e774 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:30.147893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ba316-3d04-4f1b-b0f2-6eb73719e774 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:30.149200 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 662/1317] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:30.154949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16d090-d94a-4223-a316-839308ebadd6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:30.155426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16d090-d94a-4223-a316-839308ebadd6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:30.155508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16d090-d94a-4223-a316-839308ebadd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:30.237784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ae16d090-d94a-4223-a316-839308ebadd6 None None] MFA Rules not processed for user `dd069fd5f27b41ce9e963cd415d9158d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:30.239094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ae16d090-d94a-4223-a316-839308ebadd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:30.297256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:30.343034 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 656/1318] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:30.623593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ba316-3d04-4f1b-b0f2-6eb73719e774 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:30.625166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a1ba316-3d04-4f1b-b0f2-6eb73719e774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:30.738955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ba316-3d04-4f1b-b0f2-6eb73719e774 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:30.741946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:30.820361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:30.821257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:30.821659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:30.822054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:30.824710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:30.948388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:30.949763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7940e94-bfba-4383-a246-3841c94f19bf service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:30.972086 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 663/1319] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:47:33.860760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c72b8-ba5e-4f3a-9d67-eecbf5e953f2 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:33.861633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c72b8-ba5e-4f3a-9d67-eecbf5e953f2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:33.861633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c72b8-ba5e-4f3a-9d67-eecbf5e953f2 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:33.863806 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 657/1320] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:33.869502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39faa6-42d0-4fab-9704-61cbdbae6bb1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:33.869927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39faa6-42d0-4fab-9704-61cbdbae6bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:33.870019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39faa6-42d0-4fab-9704-61cbdbae6bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:33.914767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:33.941751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5f39faa6-42d0-4fab-9704-61cbdbae6bb1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:33.942944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5f39faa6-42d0-4fab-9704-61cbdbae6bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:34.044364 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 664/1321] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:34.339131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c72b8-ba5e-4f3a-9d67-eecbf5e953f2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:34.340701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-230c72b8-ba5e-4f3a-9d67-eecbf5e953f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:34.461209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c72b8-ba5e-4f3a-9d67-eecbf5e953f2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:34.535435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:34.536060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:34.536327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:34.536592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:34.538193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:34.545203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:34.661480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:34.662833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b85cc-5f88-4601-b897-c69c6a7242c0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:34.682611 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 658/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:37.468775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382a1f-8841-4d3c-ad50-ccf49bd2c48a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:37.469346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382a1f-8841-4d3c-ad50-ccf49bd2c48a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:37.469346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7382a1f-8841-4d3c-ad50-ccf49bd2c48a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:37.470535 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 665/1323] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:37.476311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f17be5-90a4-4c4a-8bf8-957b512de792 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:37.476763 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f17be5-90a4-4c4a-8bf8-957b512de792 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:37.476871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f17be5-90a4-4c4a-8bf8-957b512de792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:37.548369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f7f17be5-90a4-4c4a-8bf8-957b512de792 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:37.549714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f7f17be5-90a4-4c4a-8bf8-957b512de792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:37.615977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:37.660332 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 659/1324] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:37.942971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382a1f-8841-4d3c-ad50-ccf49bd2c48a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:37.944595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f7382a1f-8841-4d3c-ad50-ccf49bd2c48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:38.054265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7382a1f-8841-4d3c-ad50-ccf49bd2c48a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:38.067612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:38.132554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:38.133457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:38.133726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:38.133994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:38.135685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:38.255900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:38.257247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398ba6e-3522-4c4c-9e88-fce5932eb7bc service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:38.276003 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 666/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:40.459069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89523c52-30c5-40d6-958c-78f9d60b2850 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:40.459641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89523c52-30c5-40d6-958c-78f9d60b2850 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:40.459641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89523c52-30c5-40d6-958c-78f9d60b2850 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:40.460877 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 660/1326] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:40.466280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be4d10-78b1-400f-a2aa-ba27e90ea101 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:40.466751 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be4d10-78b1-400f-a2aa-ba27e90ea101 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:40.466847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be4d10-78b1-400f-a2aa-ba27e90ea101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:40.516972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:40.536289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c6be4d10-78b1-400f-a2aa-ba27e90ea101 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:40.537501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6be4d10-78b1-400f-a2aa-ba27e90ea101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:40.642033 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 667/1327] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:40.919761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89523c52-30c5-40d6-958c-78f9d60b2850 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:40.921369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-89523c52-30c5-40d6-958c-78f9d60b2850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:41.028592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89523c52-30c5-40d6-958c-78f9d60b2850 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:41.103228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:41.104022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:41.104294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:41.104554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:41.106126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:41.124865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:41.220332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:41.221681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f4795-c3a0-46d8-afa7-783f4c3cf388 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:41.239643 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 661/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:43.461214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e0854-049f-429e-bcd9-3f0b0c049d94 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:43.461738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e0854-049f-429e-bcd9-3f0b0c049d94 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:43.461738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e0854-049f-429e-bcd9-3f0b0c049d94 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:43.463016 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 668/1329] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:47:43.468507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2107259-fad1-4512-b741-e3c8708b2c59 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:43.468963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2107259-fad1-4512-b741-e3c8708b2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:43.469068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2107259-fad1-4512-b741-e3c8708b2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:43.540681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e2107259-fad1-4512-b741-e3c8708b2c59 None None] MFA Rules not processed for user `dd069fd5f27b41ce9e963cd415d9158d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:43.541923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e2107259-fad1-4512-b741-e3c8708b2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:43.615866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:43.648417 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 662/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:43.930011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e0854-049f-429e-bcd9-3f0b0c049d94 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:43.931678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dc7e0854-049f-429e-bcd9-3f0b0c049d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:44.038438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e0854-049f-429e-bcd9-3f0b0c049d94 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:44.060159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:44.114987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:44.115634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:44.115900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:44.116157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:44.117752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:44.233896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:44.235234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337892f3-c9b6-498f-8fc4-66cdc8f29bd9 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:44.253760 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 669/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:46.614071 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a886c-c8c4-4213-bebf-11aa534a8859 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:46.614693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a886c-c8c4-4213-bebf-11aa534a8859 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:46.614693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a886c-c8c4-4213-bebf-11aa534a8859 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:46.616017 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 663/1332] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:46.622369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f210d6-9c97-4d67-b846-7b365ac0ac42 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:46.623066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f210d6-9c97-4d67-b846-7b365ac0ac42 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:46.623275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f210d6-9c97-4d67-b846-7b365ac0ac42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:46.698037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:46.700966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-58f210d6-9c97-4d67-b846-7b365ac0ac42 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:46.702791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-58f210d6-9c97-4d67-b846-7b365ac0ac42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:46.800187 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 670/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:47.096941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a886c-c8c4-4213-bebf-11aa534a8859 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:47.098961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fd0a886c-c8c4-4213-bebf-11aa534a8859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:47.218519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a886c-c8c4-4213-bebf-11aa534a8859 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:47.305506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:47.306107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:47.306377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:47.306640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:47.308275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:47.335628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:47.431231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:47.432585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee16d46-2b15-493d-847b-6641f608811e service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:47.450764 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 664/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:49.648326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7762bad-5b29-4832-a6f4-785574f9e71f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:49.648893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7762bad-5b29-4832-a6f4-785574f9e71f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:49.648893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7762bad-5b29-4832-a6f4-785574f9e71f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:49.650391 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 671/1335] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:49.656330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb97824-faf6-449e-ac7f-ba8ee84dcb89 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:49.656882 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb97824-faf6-449e-ac7f-ba8ee84dcb89 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:49.656882 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb97824-faf6-449e-ac7f-ba8ee84dcb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:49.724987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8fb97824-faf6-449e-ac7f-ba8ee84dcb89 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:49.726176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8fb97824-faf6-449e-ac7f-ba8ee84dcb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:49.804775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:49.838415 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 665/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:50.122915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7762bad-5b29-4832-a6f4-785574f9e71f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:50.124532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d7762bad-5b29-4832-a6f4-785574f9e71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:50.241690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7762bad-5b29-4832-a6f4-785574f9e71f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:50.273394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:50.320124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:50.320730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:50.321001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:50.321266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:50.322870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:50.443631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:50.445639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac02cd-683d-4b39-a974-d3a7b11c055a service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:50.466671 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 672/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:52.938625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e25f59-68ee-45cb-adb5-cc1d23f62d8d None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:52.939359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e25f59-68ee-45cb-adb5-cc1d23f62d8d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:52.939359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e25f59-68ee-45cb-adb5-cc1d23f62d8d None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:52.940553 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 666/1338] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:52.946489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d57d9-110d-4162-a283-0346c4431ae3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:52.947292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d57d9-110d-4162-a283-0346c4431ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:52.947465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d57d9-110d-4162-a283-0346c4431ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:53.027938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cf4d57d9-110d-4162-a283-0346c4431ae3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:53.029163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cf4d57d9-110d-4162-a283-0346c4431ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:53.037915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:53.132372 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 673/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:53.410456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e25f59-68ee-45cb-adb5-cc1d23f62d8d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:53.413026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-33e25f59-68ee-45cb-adb5-cc1d23f62d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:53.527526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e25f59-68ee-45cb-adb5-cc1d23f62d8d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:53.610163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:53.610773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:53.611057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:53.611342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:53.612960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:53.649187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:53.725589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:53.726941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97dffd-a520-4a80-8b3b-9c766f57ca0f service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:53.745982 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 667/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:47:57.836120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554454b1-c4ec-4e4a-a9ab-dc52785b47a8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:57.836678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554454b1-c4ec-4e4a-a9ab-dc52785b47a8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:57.836678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554454b1-c4ec-4e4a-a9ab-dc52785b47a8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:57.837941 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 674/1341] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:47:57.843301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e512-b580-4470-8e96-5897cfca6c2c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:57.843722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e512-b580-4470-8e96-5897cfca6c2c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:57.843864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501e512-b580-4470-8e96-5897cfca6c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:57.914700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7501e512-b580-4470-8e96-5897cfca6c2c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:47:57.915859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7501e512-b580-4470-8e96-5897cfca6c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:58.003000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:58.014701 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 668/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:47:58.292369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554454b1-c4ec-4e4a-a9ab-dc52785b47a8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:47:58.293951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-554454b1-c4ec-4e4a-a9ab-dc52785b47a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:58.401685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554454b1-c4ec-4e4a-a9ab-dc52785b47a8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:47:58.442537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:47:58.481297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:47:58.481901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:47:58.482171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:47:58.482437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:47:58.484192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:47:58.595949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:47:58.597411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a5a28-020a-440c-8770-3844b7beaad6 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:47:58.616922 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 675/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:01.586377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0936ff-1644-403a-afae-9735683f7485 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:01.586949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0936ff-1644-403a-afae-9735683f7485 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:01.586949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0936ff-1644-403a-afae-9735683f7485 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:01.588485 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 669/1344] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:01.593645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cc5b3-5b6f-4abf-b6ed-cebb1851b862 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:01.594147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cc5b3-5b6f-4abf-b6ed-cebb1851b862 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:01.594147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cc5b3-5b6f-4abf-b6ed-cebb1851b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:01.666184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f48cc5b3-5b6f-4abf-b6ed-cebb1851b862 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:01.667632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f48cc5b3-5b6f-4abf-b6ed-cebb1851b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:01.687893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:01.773527 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 676/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:02.050213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0936ff-1644-403a-afae-9735683f7485 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:02.052875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6c0936ff-1644-403a-afae-9735683f7485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:02.162001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0936ff-1644-403a-afae-9735683f7485 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:02.252709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:02.253325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:02.253581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:02.253846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:02.255449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:02.301459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:02.377141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:02.378784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77508ee7-0b7a-4394-afb7-b62b6a601a25 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:02.405891 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 670/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:06.796164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d65aa9-c189-44f0-8cb5-ac3fa830b088 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:06.796704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d65aa9-c189-44f0-8cb5-ac3fa830b088 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:06.796704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d65aa9-c189-44f0-8cb5-ac3fa830b088 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:06.798043 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 677/1347] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:06.804243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45f51b-1ac9-4826-b2a7-8e04b53bae0c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:06.804668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45f51b-1ac9-4826-b2a7-8e04b53bae0c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:06.804838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45f51b-1ac9-4826-b2a7-8e04b53bae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:06.878453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8e45f51b-1ac9-4826-b2a7-8e04b53bae0c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:06.879937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8e45f51b-1ac9-4826-b2a7-8e04b53bae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:06.977679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:06.980457 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 671/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:07.270188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d65aa9-c189-44f0-8cb5-ac3fa830b088 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:07.271788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-31d65aa9-c189-44f0-8cb5-ac3fa830b088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:07.378321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d65aa9-c189-44f0-8cb5-ac3fa830b088 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:07.439121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:07.466864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:07.467527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:07.467805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:07.468076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:07.469699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:07.592807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:07.594307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f503f903-6c38-4621-b43f-ef82aa1b0574 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:07.614037 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 678/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:12.516649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae087e5-ce0b-4f12-a92e-852e35cc1408 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:12.517407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae087e5-ce0b-4f12-a92e-852e35cc1408 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:12.517855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae087e5-ce0b-4f12-a92e-852e35cc1408 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:12.519949 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 672/1350] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:48:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:12.524718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee26c4-7929-4959-a1dd-39d6cf2a7ae6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:12.525464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee26c4-7929-4959-a1dd-39d6cf2a7ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:12.525884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee26c4-7929-4959-a1dd-39d6cf2a7ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:12.659361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8fee26c4-7929-4959-a1dd-39d6cf2a7ae6 None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:12.659361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8fee26c4-7929-4959-a1dd-39d6cf2a7ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:12.681848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:12.782167 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 679/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:48:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:16.861400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b45988-92e5-44c0-8522-737fb3a9d875 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:16.862291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b45988-92e5-44c0-8522-737fb3a9d875 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:16.862562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b45988-92e5-44c0-8522-737fb3a9d875 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:16.864029 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 673/1352] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:16.869287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1b10-d8a7-462c-a07c-32fd7b91ced2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:16.869739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1b10-d8a7-462c-a07c-32fd7b91ced2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:16.869887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b1b10-d8a7-462c-a07c-32fd7b91ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:16.952693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-873b1b10-d8a7-462c-a07c-32fd7b91ced2 None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:16.953986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-873b1b10-d8a7-462c-a07c-32fd7b91ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:17.028407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:17.066930 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 680/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:48:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:48:18.361959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7b07-ef6f-497e-aa64-527c5f9ad131 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:18.362419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7b07-ef6f-497e-aa64-527c5f9ad131 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:18.362508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b7b07-ef6f-497e-aa64-527c5f9ad131 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:18.363848 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 674/1354] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:18.369010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97374ad6-c3eb-4b5f-bc86-1697971bfd8e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:18.369435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97374ad6-c3eb-4b5f-bc86-1697971bfd8e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:18.369559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97374ad6-c3eb-4b5f-bc86-1697971bfd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:18.448424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-97374ad6-c3eb-4b5f-bc86-1697971bfd8e None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:18.449536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-97374ad6-c3eb-4b5f-bc86-1697971bfd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:18.557517 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 681/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:48:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:22.165233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc35b-35ec-49d5-ad44-f00263adb8f3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:22.165745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc35b-35ec-49d5-ad44-f00263adb8f3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:22.165826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dc35b-35ec-49d5-ad44-f00263adb8f3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:22.167368 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 675/1356] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:22.173425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5ef2a-fe1c-4afc-9492-e21cbf0e92bf None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:22.174391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5ef2a-fe1c-4afc-9492-e21cbf0e92bf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:22.175043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5ef2a-fe1c-4afc-9492-e21cbf0e92bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:22.216383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:22.251491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a2b5ef2a-fe1c-4afc-9492-e21cbf0e92bf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:22.252673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a2b5ef2a-fe1c-4afc-9492-e21cbf0e92bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:22.353193 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 682/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:48:22.664424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16071ebf-adad-41d5-ae29-4fdb629272f0 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:22.665181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16071ebf-adad-41d5-ae29-4fdb629272f0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:22.666142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16071ebf-adad-41d5-ae29-4fdb629272f0 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:22.669170 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 676/1358] 10.222.0.23 () {54 vars in 962 bytes} [Sun Sep 11 00:48:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:22.675142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcbc60-3d24-46b5-90e9-9041a533b5fa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:22.675505 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcbc60-3d24-46b5-90e9-9041a533b5fa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:22.675775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcbc60-3d24-46b5-90e9-9041a533b5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:22.755313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-89bcbc60-3d24-46b5-90e9-9041a533b5fa None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:22.756503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-89bcbc60-3d24-46b5-90e9-9041a533b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:22.809036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:22.878252 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 683/1359] 10.222.0.23 () {58 vars in 1059 bytes} [Sun Sep 11 00:48:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:22.886802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16071ebf-adad-41d5-ae29-4fdb629272f0 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:22.888398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-16071ebf-adad-41d5-ae29-4fdb629272f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:23.048723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16071ebf-adad-41d5-ae29-4fdb629272f0 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:23.132374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:23.179621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:23.180256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:23.180514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:23.180771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:23.182475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:23.282286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:23.283977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028849cf-8b47-4337-9e12-0a0c42045671 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:23.313789 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 677/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:25.670893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834678-44e1-4efe-91ac-47adcb70287a None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:25.671387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834678-44e1-4efe-91ac-47adcb70287a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:25.671902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09834678-44e1-4efe-91ac-47adcb70287a None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:25.673905 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 684/1361] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:25.679783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999863-ddcc-4698-9adc-2ac47a58775c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:25.679783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999863-ddcc-4698-9adc-2ac47a58775c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:25.680625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84999863-ddcc-4698-9adc-2ac47a58775c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:25.747735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-84999863-ddcc-4698-9adc-2ac47a58775c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:25.748951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-84999863-ddcc-4698-9adc-2ac47a58775c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:25.787949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:25.851769 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 678/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:26.136057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834678-44e1-4efe-91ac-47adcb70287a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:26.137605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-09834678-44e1-4efe-91ac-47adcb70287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:26.312691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09834678-44e1-4efe-91ac-47adcb70287a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:26.325192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:26.458929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:26.459591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:26.459866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:26.460139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:26.461788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:26.550695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:26.552728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d733bc02-97b6-4bd2-a20c-5242a4ccaf10 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:26.570769 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 685/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:29.039783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6574ff-b618-4300-8c0a-1c84fd029cde None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:29.040772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6574ff-b618-4300-8c0a-1c84fd029cde None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:29.041105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6574ff-b618-4300-8c0a-1c84fd029cde None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:29.042668 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 679/1364] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:29.048258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c71980-ba6d-4f73-9ae8-d963ab338787 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:29.048690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c71980-ba6d-4f73-9ae8-d963ab338787 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:29.048867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c71980-ba6d-4f73-9ae8-d963ab338787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:29.065854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:29.121706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-37c71980-ba6d-4f73-9ae8-d963ab338787 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:29.122929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-37c71980-ba6d-4f73-9ae8-d963ab338787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:29.252355 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 686/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:29.537240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4993fc9-f48f-4bb0-94a0-dc47d86fcd3b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:29.537640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4993fc9-f48f-4bb0-94a0-dc47d86fcd3b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:29.538036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4993fc9-f48f-4bb0-94a0-dc47d86fcd3b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:29.539709 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 680/1366] 10.222.0.23 () {54 vars in 962 bytes} [Sun Sep 11 00:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:29.545257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7b561-511d-4f5e-a62c-9245d6880ab0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:29.545542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7b561-511d-4f5e-a62c-9245d6880ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:29.545815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd7b561-511d-4f5e-a62c-9245d6880ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:29.629490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2dd7b561-511d-4f5e-a62c-9245d6880ab0 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:29.630951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2dd7b561-511d-4f5e-a62c-9245d6880ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:29.668395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:29.765169 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 687/1367] 10.222.0.23 () {58 vars in 1059 bytes} [Sun Sep 11 00:48:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:29.776226 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4993fc9-f48f-4bb0-94a0-dc47d86fcd3b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:29.778138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a4993fc9-f48f-4bb0-94a0-dc47d86fcd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:29.935873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4993fc9-f48f-4bb0-94a0-dc47d86fcd3b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:29.959014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:30.059500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:30.060172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:30.060270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:30.060542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:30.062191 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:30.169819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:30.172603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea1b8b-ef86-4d1d-986f-751bc31eda44 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:30.191476 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 681/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:30.226553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513bbef-8694-4156-9e56-1467077ba367 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:30.227156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513bbef-8694-4156-9e56-1467077ba367 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:30.227445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513bbef-8694-4156-9e56-1467077ba367 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:30.229455 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 688/1369] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:30.235889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100a78-8ab6-4543-804f-e10c345ea0ae None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:30.236303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100a78-8ab6-4543-804f-e10c345ea0ae None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:30.236684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100a78-8ab6-4543-804f-e10c345ea0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:30.271213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:30.312506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c8100a78-8ab6-4543-804f-e10c345ea0ae None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:30.313757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8100a78-8ab6-4543-804f-e10c345ea0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:30.442196 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 682/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:30.769868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577bbf-75f7-4852-afe9-f1028a327880 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:30.770149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577bbf-75f7-4852-afe9-f1028a327880 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:30.770419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44577bbf-75f7-4852-afe9-f1028a327880 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:30.772038 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 689/1371] 10.222.0.23 () {54 vars in 964 bytes} [Sun Sep 11 00:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 00:48:30.776774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05416a-5a79-42ac-9904-26755ec03d33 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:30.777053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05416a-5a79-42ac-9904-26755ec03d33 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:30.777326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05416a-5a79-42ac-9904-26755ec03d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:30.866345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5b05416a-5a79-42ac-9904-26755ec03d33 None None] MFA Rules not processed for user `7d71b946e7d64a50888cfcc337d688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:30.867839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5b05416a-5a79-42ac-9904-26755ec03d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:30.921250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:30.955259 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 683/1372] 10.222.0.23 () {58 vars in 1061 bytes} [Sun Sep 11 00:48:30 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:30.963371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44577bbf-75f7-4852-afe9-f1028a327880 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:30.965087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-44577bbf-75f7-4852-afe9-f1028a327880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:31.078766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44577bbf-75f7-4852-afe9-f1028a327880 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:31.128936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:31.186760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:31.187431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:31.187727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:31.188003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:31.189610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:31.432842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:31.434403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6d6f56-47cf-4799-95e2-c1c06ca33fc8 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:31.454134 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 690/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:32.429240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388af231-3dd0-4056-9747-6c2ba9f140a1 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:32.429862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388af231-3dd0-4056-9747-6c2ba9f140a1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:32.429862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388af231-3dd0-4056-9747-6c2ba9f140a1 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:32.431095 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 684/1374] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:32.436588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0232827-18a1-41e6-9312-d30d2cec1e9e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:32.437014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0232827-18a1-41e6-9312-d30d2cec1e9e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:32.437111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0232827-18a1-41e6-9312-d30d2cec1e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:32.503592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a0232827-18a1-41e6-9312-d30d2cec1e9e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:32.504809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a0232827-18a1-41e6-9312-d30d2cec1e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:32.526283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:32.617666 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 691/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:32.898597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388af231-3dd0-4056-9747-6c2ba9f140a1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:32.900220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-388af231-3dd0-4056-9747-6c2ba9f140a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:33.067659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388af231-3dd0-4056-9747-6c2ba9f140a1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:33.122931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:33.188719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:33.189332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:33.189602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:33.189864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:33.191581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:33.284736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:33.286322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd58e3-5eef-4c66-95f6-ce3cc775529b service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:33.305424 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 685/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:35.520388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b884e-57be-4187-b6aa-4738177d34aa None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:35.520959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b884e-57be-4187-b6aa-4738177d34aa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:35.520959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b884e-57be-4187-b6aa-4738177d34aa None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:35.522348 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 692/1377] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:35.527640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42980c-3bd2-4108-b1b5-ff20243b2ad3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:35.528102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42980c-3bd2-4108-b1b5-ff20243b2ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:35.528200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42980c-3bd2-4108-b1b5-ff20243b2ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:35.606799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-be42980c-3bd2-4108-b1b5-ff20243b2ad3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:35.608292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-be42980c-3bd2-4108-b1b5-ff20243b2ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:35.616211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:35.718020 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 686/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:36.016481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b884e-57be-4187-b6aa-4738177d34aa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:36.018063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a42b884e-57be-4187-b6aa-4738177d34aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:36.173832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b884e-57be-4187-b6aa-4738177d34aa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:36.178258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:36.304410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:36.304921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:36.305199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:36.305465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:36.307156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:36.403086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:36.404729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0401dd0-49ff-4572-b326-d78c15b0c8aa service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:36.423699 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 693/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:38.626830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6660-d5d5-40f9-8f96-111e8a9564de None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:38.627730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6660-d5d5-40f9-8f96-111e8a9564de None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:38.627730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df6660-d5d5-40f9-8f96-111e8a9564de None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:38.629011 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 687/1380] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:38.634911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91a624-b947-4adb-b1f1-4fd5a72daae1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:38.635405 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91a624-b947-4adb-b1f1-4fd5a72daae1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:38.635516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91a624-b947-4adb-b1f1-4fd5a72daae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:38.647862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:38.711327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ab91a624-b947-4adb-b1f1-4fd5a72daae1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:38.712656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ab91a624-b947-4adb-b1f1-4fd5a72daae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:38.826742 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 694/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:39.117612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df6660-d5d5-40f9-8f96-111e8a9564de None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:39.119231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f6df6660-d5d5-40f9-8f96-111e8a9564de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:39.282449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:39.293816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df6660-d5d5-40f9-8f96-111e8a9564de None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:39.422369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:39.422912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:39.423208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:39.423500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:39.425154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:39.517833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:39.519729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd1e00-1f31-4fc5-8753-fa711ff3b964 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:39.530386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:39.540213 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 688/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:42.398614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc2fdd-ae1d-4155-be96-a06b8ea026f7 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:42.399147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc2fdd-ae1d-4155-be96-a06b8ea026f7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:42.399147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc2fdd-ae1d-4155-be96-a06b8ea026f7 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:42.400720 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 695/1383] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:42.407900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f495b-b876-4465-9736-15274e4715af None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:42.408359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f495b-b876-4465-9736-15274e4715af None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:42.408618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f495b-b876-4465-9736-15274e4715af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:42.481927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-513f495b-b876-4465-9736-15274e4715af None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:42.483165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-513f495b-b876-4465-9736-15274e4715af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:42.585267 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 689/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:42.966028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc2fdd-ae1d-4155-be96-a06b8ea026f7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:42.968285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6afc2fdd-ae1d-4155-be96-a06b8ea026f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:43.078262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:43.142993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc2fdd-ae1d-4155-be96-a06b8ea026f7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:43.268525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:43.269277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:43.269657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:43.270012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:43.271842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:43.325739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:43.369702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:43.371514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ffeda-c7ad-4df9-818f-a039dc0df84f service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:43.390387 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 696/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:45.559537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f43c-a2ea-4482-b21d-8050f1c980ae None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:45.560078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f43c-a2ea-4482-b21d-8050f1c980ae None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:45.560078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2f43c-a2ea-4482-b21d-8050f1c980ae None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:45.561355 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 690/1386] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:45.567197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9e15-95a1-4944-8fe1-26179034d3ab None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:45.567598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9e15-95a1-4944-8fe1-26179034d3ab None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:45.567811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9e15-95a1-4944-8fe1-26179034d3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:45.637537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-52ae9e15-95a1-4944-8fe1-26179034d3ab None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:45.638775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-52ae9e15-95a1-4944-8fe1-26179034d3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:45.741498 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 697/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:46.024477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f43c-a2ea-4482-b21d-8050f1c980ae None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:46.026101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a4f2f43c-a2ea-4482-b21d-8050f1c980ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:46.105410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:46.180849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2f43c-a2ea-4482-b21d-8050f1c980ae None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:46.296290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:46.297186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:46.297449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:46.297716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:46.299330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:46.340549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:46.396979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:46.398569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641b863-02dc-464a-80ab-171eddd8f2f4 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:46.417274 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 691/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:46.574592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59cd40-0adf-4ee0-ac23-fd85ba2f5624 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:46.575146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59cd40-0adf-4ee0-ac23-fd85ba2f5624 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:46.575146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59cd40-0adf-4ee0-ac23-fd85ba2f5624 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:46.576593 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 698/1389] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:46.583023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ea61a-bfca-474f-bcab-7c17dd0d8895 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:46.583649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ea61a-bfca-474f-bcab-7c17dd0d8895 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:46.583871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ea61a-bfca-474f-bcab-7c17dd0d8895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:46.659957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d51ea61a-bfca-474f-bcab-7c17dd0d8895 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:46.661254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d51ea61a-bfca-474f-bcab-7c17dd0d8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:46.766318 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 692/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:48:47.060255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdac53b-a677-4006-afaf-b232d10ae58f None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:47.060560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdac53b-a677-4006-afaf-b232d10ae58f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:47.060838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdac53b-a677-4006-afaf-b232d10ae58f None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:47.062289 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 699/1391] 10.222.0.23 () {54 vars in 966 bytes} [Sun Sep 11 00:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:47.067895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f56b25-3930-4ceb-aa9b-3aebaf98fa78 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:47.068177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f56b25-3930-4ceb-aa9b-3aebaf98fa78 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:47.068446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f56b25-3930-4ceb-aa9b-3aebaf98fa78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:47.088162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:47.164003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b3f56b25-3930-4ceb-aa9b-3aebaf98fa78 None None] MFA Rules not processed for user `8388bedc588f4f5f94f1e732d46e83b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:47.165245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3f56b25-3930-4ceb-aa9b-3aebaf98fa78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:47.243832 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 693/1392] 10.222.0.23 () {58 vars in 1063 bytes} [Sun Sep 11 00:48:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:47.252943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdac53b-a677-4006-afaf-b232d10ae58f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:47.254520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7bdac53b-a677-4006-afaf-b232d10ae58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:47.286471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:47.365537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdac53b-a677-4006-afaf-b232d10ae58f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:47.443763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:47.444361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:47.444630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:47.444948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:47.446565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:47.524222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:47.557381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:47.558736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d1561-f2a6-4f04-a0fe-2f672146c07a service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:47.577908 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 700/1393] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:48.327760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe96c1c-5b44-4235-8d68-c36ba76a7ed3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:48.328246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe96c1c-5b44-4235-8d68-c36ba76a7ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:48.328340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe96c1c-5b44-4235-8d68-c36ba76a7ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:48.400197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-cfe96c1c-5b44-4235-8d68-c36ba76a7ed3 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:48.401383 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cfe96c1c-5b44-4235-8d68-c36ba76a7ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:48.456076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:48.488714 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 694/1394] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:48:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:48.536576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655357d-ef58-46f0-ac43-c2ac15f1ae03 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:48.536924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655357d-ef58-46f0-ac43-c2ac15f1ae03 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:48.537270 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655357d-ef58-46f0-ac43-c2ac15f1ae03 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:48.538738 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 701/1395] 10.222.0.23 () {54 vars in 943 bytes} [Sun Sep 11 00:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:48.545254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:48.545536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:48.545806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:48.610334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaa81-f989-4a06-a4fe-42a021a9048b None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:48.610907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaa81-f989-4a06-a4fe-42a021a9048b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:48.610907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3deaa81-f989-4a06-a4fe-42a021a9048b None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:48.612605 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 702/1396] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:48.617639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:48.617912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:48.618171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:48.625768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] MFA Rules not processed for user `512080b44c7542439333bd8b5f941496`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:48.626979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:48.685322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:48.686539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:48.738853 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 695/1397] 10.222.0.23 () {58 vars in 1040 bytes} [Sun Sep 11 00:48:48 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:48.748398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:48.760457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d17306-5103-4f86-b773-1f2cd4311f8b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:48.786973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:48.788072 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 703/1398] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:48.868576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:48.869214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:48.869482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:48.869741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:48.871398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:48.974968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:48.976692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80f1040-43a1-4b8b-8468-02fadef7a957 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:48.985166 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 696/1399] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:48:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 00:48:49.072803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:49.074002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:49.119258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:49.203138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568f9a-b266-4d9d-8c20-229b0a5c6428 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:49.325643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:49.326256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:49.326518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:49.326780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:49.328446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:49.366855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:49.429557 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:49.431469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f17222-a056-4c3d-ae11-1e39e5eabe29 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:49.451020 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 704/1400] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:51.664493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13743-24e5-4ac5-840a-581069dcaf91 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:51.665246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13743-24e5-4ac5-840a-581069dcaf91 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:51.665246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd13743-24e5-4ac5-840a-581069dcaf91 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:51.666758 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 697/1401] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:51.672191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389595f-a653-4045-bf6b-0b78227063c8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:51.672633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389595f-a653-4045-bf6b-0b78227063c8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:51.672737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389595f-a653-4045-bf6b-0b78227063c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:51.740621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6389595f-a653-4045-bf6b-0b78227063c8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:51.742676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6389595f-a653-4045-bf6b-0b78227063c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:51.847556 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 705/1402] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:52.140583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd13743-24e5-4ac5-840a-581069dcaf91 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:52.142170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1dd13743-24e5-4ac5-840a-581069dcaf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:52.216237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:52.316180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd13743-24e5-4ac5-840a-581069dcaf91 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:52.480420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:52.481032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:52.481809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:52.481809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:52.483283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:52.505445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:52.591691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:52.593296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06623-0fd6-41de-9285-c16b1cd6f249 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:52.629043 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 698/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:54.944982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec20be-49e2-4130-aefb-b8fe754f1e1c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:54.945605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec20be-49e2-4130-aefb-b8fe754f1e1c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:54.945605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec20be-49e2-4130-aefb-b8fe754f1e1c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:54.947029 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 706/1404] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:54.952140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126a43f-9941-449e-badc-0b2efb76f2db None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:54.952568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126a43f-9941-449e-badc-0b2efb76f2db None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:54.952651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126a43f-9941-449e-badc-0b2efb76f2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:55.021334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c126a43f-9941-449e-badc-0b2efb76f2db None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:55.022484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c126a43f-9941-449e-badc-0b2efb76f2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:55.128007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:55.135217 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 699/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:55.422473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec20be-49e2-4130-aefb-b8fe754f1e1c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:55.424065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-16ec20be-49e2-4130-aefb-b8fe754f1e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:55.434742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:55.587079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec20be-49e2-4130-aefb-b8fe754f1e1c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:55.675853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:55.710835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:55.711487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:55.711757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:55.712023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:55.713637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:55.810747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:55.814573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01a65a-23dd-49bd-b0d9-1624a6f22b5b service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:55.835266 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 707/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:48:58.078256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56956a6a-58bc-411e-a2e7-1ea30192c486 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:58.078831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56956a6a-58bc-411e-a2e7-1ea30192c486 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:58.078831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56956a6a-58bc-411e-a2e7-1ea30192c486 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:58.086016 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 700/1407] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:48:58.088565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051e937-749f-4b3a-8db8-d865643aae56 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:58.089444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051e937-749f-4b3a-8db8-d865643aae56 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:58.089937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051e937-749f-4b3a-8db8-d865643aae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:58.172362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f051e937-749f-4b3a-8db8-d865643aae56 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:48:58.173598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f051e937-749f-4b3a-8db8-d865643aae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:58.225617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:58.287224 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 708/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:48:58.575640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56956a6a-58bc-411e-a2e7-1ea30192c486 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:48:58.579132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-56956a6a-58bc-411e-a2e7-1ea30192c486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:58.744027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56956a6a-58bc-411e-a2e7-1ea30192c486 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:48:58.822214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:48:58.874130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:48:58.874748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:48:58.875019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:48:58.875341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:48:58.876961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:48:58.978488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:48:58.980116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a321bb9-b4b0-4ba4-8ac4-459a22848582 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:48:58.998791 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 701/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:48:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:08.283476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=23 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:08.996671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=64 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:11.921389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d612e-99bb-46cb-a541-880a6ee5a3f8 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:11.922024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d612e-99bb-46cb-a541-880a6ee5a3f8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:11.922024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d612e-99bb-46cb-a541-880a6ee5a3f8 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:11.923289 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 709/1410] 10.222.0.23 () {54 vars in 918 bytes} [Sun Sep 11 00:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:11.929706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969f606-f313-4fdf-8006-a3046469f4ba None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:11.930140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969f606-f313-4fdf-8006-a3046469f4ba None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:11.930253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969f606-f313-4fdf-8006-a3046469f4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:12.020941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c969f606-f313-4fdf-8006-a3046469f4ba None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:12.022151 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c969f606-f313-4fdf-8006-a3046469f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:12.134400 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 702/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Sun Sep 11 00:49:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:12.148966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eef14-63be-424b-abcd-213cabcf9b65 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:12.149247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eef14-63be-424b-abcd-213cabcf9b65 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:12.149520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eef14-63be-424b-abcd-213cabcf9b65 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:12.150905 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 710/1412] 10.222.0.23 () {54 vars in 918 bytes} [Sun Sep 11 00:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:12.155776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c1b07-d48b-41cc-a537-bf2cbe980a73 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:12.156063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c1b07-d48b-41cc-a537-bf2cbe980a73 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:12.156335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c1b07-d48b-41cc-a537-bf2cbe980a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:12.230397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:12.233465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2d0c1b07-d48b-41cc-a537-bf2cbe980a73 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:12.235120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d0c1b07-d48b-41cc-a537-bf2cbe980a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:12.367513 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 703/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Sun Sep 11 00:49:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:14.960330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b7c6-0368-4b07-8274-8fdd3bbf1951 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:14.960959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b7c6-0368-4b07-8274-8fdd3bbf1951 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:14.960959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b7c6-0368-4b07-8274-8fdd3bbf1951 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:14.962210 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:14.967845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842463a8-e496-4ae8-b806-3a8ef698a42b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:14.968291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842463a8-e496-4ae8-b806-3a8ef698a42b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:14.968406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842463a8-e496-4ae8-b806-3a8ef698a42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:15.044822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-842463a8-e496-4ae8-b806-3a8ef698a42b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:15.046061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-842463a8-e496-4ae8-b806-3a8ef698a42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:15.112896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:15.169651 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:15.468208 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b7c6-0368-4b07-8274-8fdd3bbf1951 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:15.470970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ff76b7c6-0368-4b07-8274-8fdd3bbf1951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:15.627502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b7c6-0368-4b07-8274-8fdd3bbf1951 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:15.714375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:15.759658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:15.760267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:15.760540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:15.760816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:15.762445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:15.858963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:15.860588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d959ce-df6a-4511-9562-557a73dca32c service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:15.878805 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 712/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:17.032237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e50878-add7-455e-985b-0726ec3c815f None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:17.032840 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e50878-add7-455e-985b-0726ec3c815f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:17.032840 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e50878-add7-455e-985b-0726ec3c815f None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:17.034658 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:17.041952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba3e95-d2de-4e00-81dc-108156b06bd9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:17.042473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba3e95-d2de-4e00-81dc-108156b06bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:17.042473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba3e95-d2de-4e00-81dc-108156b06bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:17.156071 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fba3e95-d2de-4e00-81dc-108156b06bd9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:17.157280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fba3e95-d2de-4e00-81dc-108156b06bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:17.186875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:17.259526 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:17.407552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e50878-add7-455e-985b-0726ec3c815f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:17.409117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f8e50878-add7-455e-985b-0726ec3c815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:17.562670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e50878-add7-455e-985b-0726ec3c815f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:17.603420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:17.699268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:17.699810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:17.700085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:17.700353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:17.701982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:17.803838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:17.805515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab3b88-bb30-4e86-aa9b-9f32b78f670e service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:17.825494 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 706/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:18.990981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc8203-5fe1-4e61-966d-cbf8deb23195 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:18.992010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc8203-5fe1-4e61-966d-cbf8deb23195 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:18.992010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc8203-5fe1-4e61-966d-cbf8deb23195 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:18.994052 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:19.000549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb57b6-94f5-4617-9f7d-174faf179712 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:19.001023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb57b6-94f5-4617-9f7d-174faf179712 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:19.001115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb57b6-94f5-4617-9f7d-174faf179712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:19.059378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:19.083918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7adb57b6-94f5-4617-9f7d-174faf179712 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:19.085181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7adb57b6-94f5-4617-9f7d-174faf179712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:19.187783 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:19.343686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc8203-5fe1-4e61-966d-cbf8deb23195 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:19.345235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c9bc8203-5fe1-4e61-966d-cbf8deb23195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:19.500059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc8203-5fe1-4e61-966d-cbf8deb23195 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:19.512834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:19.621341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:19.621985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:19.622256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:19.622517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:19.624158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:19.721074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:19.722670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a782d56-20b8-4772-810d-703ae09e7c5f service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:19.741010 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 715/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:20.863142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cb205-be2c-4d87-8ad6-2d3c1b5fb84e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:20.864072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cb205-be2c-4d87-8ad6-2d3c1b5fb84e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:20.864072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06cb205-be2c-4d87-8ad6-2d3c1b5fb84e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:20.866278 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:20.872783 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18635f0e-d5aa-489c-9aa9-06584bbd6867 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:20.873239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18635f0e-d5aa-489c-9aa9-06584bbd6867 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:20.873347 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18635f0e-d5aa-489c-9aa9-06584bbd6867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:20.894428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:20.957485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-18635f0e-d5aa-489c-9aa9-06584bbd6867 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:20.958697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-18635f0e-d5aa-489c-9aa9-06584bbd6867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:21.065995 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:21.218288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cb205-be2c-4d87-8ad6-2d3c1b5fb84e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:21.219880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f06cb205-be2c-4d87-8ad6-2d3c1b5fb84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:21.336309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:21.372105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06cb205-be2c-4d87-8ad6-2d3c1b5fb84e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:21.497944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:21.498555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:21.498824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:21.499088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:21.500728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:21.578449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:21.592453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:21.594047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a94716-76b7-4d1e-bc6b-82157537e9f9 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:21.613385 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 709/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:22.737637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d81211-4f9e-4728-b4bf-961d9bc46d7d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:22.738209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d81211-4f9e-4728-b4bf-961d9bc46d7d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:22.738209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d81211-4f9e-4728-b4bf-961d9bc46d7d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:22.739517 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:22.746063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db20c8-2514-4ebc-bb5a-dcbce8e75e34 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:22.746524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db20c8-2514-4ebc-bb5a-dcbce8e75e34 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:22.746620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db20c8-2514-4ebc-bb5a-dcbce8e75e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:22.830234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-25db20c8-2514-4ebc-bb5a-dcbce8e75e34 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:22.832200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-25db20c8-2514-4ebc-bb5a-dcbce8e75e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:22.953356 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:49:23.107101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d81211-4f9e-4728-b4bf-961d9bc46d7d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:23.109052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-56d81211-4f9e-4728-b4bf-961d9bc46d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:23.199520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:23.266009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d81211-4f9e-4728-b4bf-961d9bc46d7d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:23.398358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:23.398872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:23.399150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:23.399450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:23.401070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:23.456887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:23.500056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:23.501677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18403f10-a34d-4381-9d8c-6aa4de81af3c service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:23.520759 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 718/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:24.592356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecc830-0157-4210-8140-c103a618dfc8 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:24.593333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecc830-0157-4210-8140-c103a618dfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:24.593333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecc830-0157-4210-8140-c103a618dfc8 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:24.595488 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:24.602051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36677c-6f41-4dd4-94ad-94cccef2fee4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:24.602542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36677c-6f41-4dd4-94ad-94cccef2fee4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:24.602858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36677c-6f41-4dd4-94ad-94cccef2fee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:24.686742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3b36677c-6f41-4dd4-94ad-94cccef2fee4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:24.688573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3b36677c-6f41-4dd4-94ad-94cccef2fee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:24.792384 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:24.946302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecc830-0157-4210-8140-c103a618dfc8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:24.948621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2cecc830-0157-4210-8140-c103a618dfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:25.003005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:25.109006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecc830-0157-4210-8140-c103a618dfc8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:25.230588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:25.231107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:25.231412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:25.231668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:25.233263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:25.240269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:25.331318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:25.332897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a58f9-1c22-4089-9531-4988cfa4ad77 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:25.352076 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 712/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:26.414958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293aa6d-e773-4243-8d99-a581deebb089 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:26.415726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293aa6d-e773-4243-8d99-a581deebb089 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:26.415878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293aa6d-e773-4243-8d99-a581deebb089 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:26.417907 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:26.427236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33487b-d29a-4368-bc80-8096fceb4c26 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:26.427707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33487b-d29a-4368-bc80-8096fceb4c26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:26.427812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33487b-d29a-4368-bc80-8096fceb4c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:26.504723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8d33487b-d29a-4368-bc80-8096fceb4c26 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:26.505907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8d33487b-d29a-4368-bc80-8096fceb4c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:26.580207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:26.617500 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:26.770754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293aa6d-e773-4243-8d99-a581deebb089 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:26.772273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6293aa6d-e773-4243-8d99-a581deebb089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:26.791888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:26.928953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293aa6d-e773-4243-8d99-a581deebb089 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:27.028867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:27.063483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:27.064083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:27.064350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:27.064607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:27.066306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:27.157817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:27.159548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5808d-1329-46f3-9919-cbe5717a3860 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:27.178757 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 721/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:28.218356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73d20c-b26a-4128-90d5-16763ebe175b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:28.218902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73d20c-b26a-4128-90d5-16763ebe175b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:28.218902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73d20c-b26a-4128-90d5-16763ebe175b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:28.220293 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:28.227839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed816c-b810-4acd-bfaf-9b3c29cc568c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:28.228307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed816c-b810-4acd-bfaf-9b3c29cc568c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:28.228415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed816c-b810-4acd-bfaf-9b3c29cc568c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:28.311262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6aed816c-b810-4acd-bfaf-9b3c29cc568c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:28.312495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6aed816c-b810-4acd-bfaf-9b3c29cc568c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:28.356105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:28.415422 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:28.568810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73d20c-b26a-4128-90d5-16763ebe175b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:28.571308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cf73d20c-b26a-4128-90d5-16763ebe175b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:28.727137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73d20c-b26a-4128-90d5-16763ebe175b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:28.789189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:28.849793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:28.850395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:28.850668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:28.850935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:28.852564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:28.950459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:28.952074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db795f-0d15-4a66-8ce6-4fadd59b8deb service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:28.970185 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 715/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:30.033750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cda59-7681-4717-87dc-3dbb02338223 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:30.034371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cda59-7681-4717-87dc-3dbb02338223 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:30.034371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cda59-7681-4717-87dc-3dbb02338223 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:30.035668 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:49:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:30.043342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd795d-85d0-463e-ac6b-0470a06449d2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:30.044091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd795d-85d0-463e-ac6b-0470a06449d2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:30.044281 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd795d-85d0-463e-ac6b-0470a06449d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:30.127209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-27dd795d-85d0-463e-ac6b-0470a06449d2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:30.128455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-27dd795d-85d0-463e-ac6b-0470a06449d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:30.130481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:30.236785 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:30.388186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cda59-7681-4717-87dc-3dbb02338223 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:30.389771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4e7cda59-7681-4717-87dc-3dbb02338223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:30.553029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cda59-7681-4717-87dc-3dbb02338223 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:30.583090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:30.679683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:30.680225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:30.680499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:30.680760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:30.682409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:30.794222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:30.795861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95071d50-fbf9-4635-bfb5-adf43f3a2984 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:30.820078 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 724/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:32.871213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952ce98-df44-4388-91ec-629941d17138 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:32.871826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952ce98-df44-4388-91ec-629941d17138 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:32.871826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952ce98-df44-4388-91ec-629941d17138 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:32.873106 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:32.879916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6470-42c5-4239-943a-f2ad03dcaa91 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:32.880545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6470-42c5-4239-943a-f2ad03dcaa91 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:32.880545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6470-42c5-4239-943a-f2ad03dcaa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:32.934017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:32.961305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f6aa6470-42c5-4239-943a-f2ad03dcaa91 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:32.962537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f6aa6470-42c5-4239-943a-f2ad03dcaa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:33.095136 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:33.382868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952ce98-df44-4388-91ec-629941d17138 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:33.384482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c952ce98-df44-4388-91ec-629941d17138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:33.503054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952ce98-df44-4388-91ec-629941d17138 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:33.537139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:33.583025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:33.583649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:33.583917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:33.584180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:33.585771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:33.697506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:33.698940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae41e9-1466-48aa-9118-d09a6ef48b22 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:33.717497 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 718/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:36.020616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26d103-70ed-4df0-915c-9edfeb50d462 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:36.021225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26d103-70ed-4df0-915c-9edfeb50d462 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:36.021225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a26d103-70ed-4df0-915c-9edfeb50d462 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:36.022521 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:36.029125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07143ab6-216e-4bed-9e9a-dfbab63d8f54 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:36.029882 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07143ab6-216e-4bed-9e9a-dfbab63d8f54 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:36.030038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07143ab6-216e-4bed-9e9a-dfbab63d8f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:36.108242 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-07143ab6-216e-4bed-9e9a-dfbab63d8f54 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:36.109532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-07143ab6-216e-4bed-9e9a-dfbab63d8f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:36.123143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:36.210715 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:36.492659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26d103-70ed-4df0-915c-9edfeb50d462 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:36.495390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a26d103-70ed-4df0-915c-9edfeb50d462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:36.606852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a26d103-70ed-4df0-915c-9edfeb50d462 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:36.619572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:36.685475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:36.686004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:36.686276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:36.686544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:36.688202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:36.813748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:36.815216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf878615-02e1-400f-8441-5d60592d6a0c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:36.834410 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 727/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:39.376404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b03a77-3769-47d8-a1c7-9aa2ef7cca83 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:39.377016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b03a77-3769-47d8-a1c7-9aa2ef7cca83 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:39.377016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b03a77-3769-47d8-a1c7-9aa2ef7cca83 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:39.378426 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:39.384501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a8384-d67a-4775-b261-1baaff9c1935 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:39.385086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a8384-d67a-4775-b261-1baaff9c1935 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:39.385214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a8384-d67a-4775-b261-1baaff9c1935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:39.444415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:39.463901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b67a8384-d67a-4775-b261-1baaff9c1935 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:39.465173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b67a8384-d67a-4775-b261-1baaff9c1935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:39.576702 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:39.860201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b03a77-3769-47d8-a1c7-9aa2ef7cca83 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:39.861784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-10b03a77-3769-47d8-a1c7-9aa2ef7cca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:39.976838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b03a77-3769-47d8-a1c7-9aa2ef7cca83 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:40.018266 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:40.054832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:40.055468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:40.055737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:40.056000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:40.057561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:40.180709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:40.182143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e013c9e-6d1f-405d-99f0-7b3daf7c683d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:40.200506 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 721/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:42.455385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178c2a0-aa8a-4cf0-8cbb-efd8e0ab9ded None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:42.455942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178c2a0-aa8a-4cf0-8cbb-efd8e0ab9ded None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:42.455942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178c2a0-aa8a-4cf0-8cbb-efd8e0ab9ded None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:42.457357 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:42.463797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43dc97-34e7-4f97-9aa7-73880cd0b859 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:42.464266 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43dc97-34e7-4f97-9aa7-73880cd0b859 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:42.464392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43dc97-34e7-4f97-9aa7-73880cd0b859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:42.535498 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bf43dc97-34e7-4f97-9aa7-73880cd0b859 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:42.536740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bf43dc97-34e7-4f97-9aa7-73880cd0b859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:42.558731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:42.644263 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:42.929559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178c2a0-aa8a-4cf0-8cbb-efd8e0ab9ded None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:42.931114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f178c2a0-aa8a-4cf0-8cbb-efd8e0ab9ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:43.051220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178c2a0-aa8a-4cf0-8cbb-efd8e0ab9ded None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:43.078147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:43.133855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:43.134462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:43.134729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:43.134991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:43.136657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:43.259542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:43.260898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b495d6e-06ca-4f5f-b87f-5ccd627c7acd service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:43.278889 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 730/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:49:45.462877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e501e9e-0882-4a0e-9422-a0de635eac7b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:45.463795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e501e9e-0882-4a0e-9422-a0de635eac7b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:45.463795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e501e9e-0882-4a0e-9422-a0de635eac7b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:45.465888 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:45.473895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4e90-a888-4084-b0d0-0074ea4bbb32 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:45.474300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4e90-a888-4084-b0d0-0074ea4bbb32 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:45.474417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec4e90-a888-4084-b0d0-0074ea4bbb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:45.540796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:45.544022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-46ec4e90-a888-4084-b0d0-0074ea4bbb32 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:45.545890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-46ec4e90-a888-4084-b0d0-0074ea4bbb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:45.644982 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:45.930248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e501e9e-0882-4a0e-9422-a0de635eac7b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:45.932799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1e501e9e-0882-4a0e-9422-a0de635eac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:46.061245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e501e9e-0882-4a0e-9422-a0de635eac7b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:46.113896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:46.140027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:46.140638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:46.140914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:46.141175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:46.142795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:46.257211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:46.258568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306e3d8-a108-40a5-b3e0-92b077114737 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:46.281655 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 724/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:48.505119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9decb-70c0-4a13-8884-74b17d0b706c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:48.505707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9decb-70c0-4a13-8884-74b17d0b706c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:48.505707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9decb-70c0-4a13-8884-74b17d0b706c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:48.507041 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:48.513602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb289010-ef26-478a-a89f-52391f4a6bad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:48.514378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb289010-ef26-478a-a89f-52391f4a6bad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:48.514553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb289010-ef26-478a-a89f-52391f4a6bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:48.592077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bb289010-ef26-478a-a89f-52391f4a6bad None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:48.593335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb289010-ef26-478a-a89f-52391f4a6bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:48.633620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:48.710331 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:49.004676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9decb-70c0-4a13-8884-74b17d0b706c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:49.007040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-91c9decb-70c0-4a13-8884-74b17d0b706c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:49.122504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9decb-70c0-4a13-8884-74b17d0b706c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:49.155641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:49.204630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:49.205230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:49.205475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:49.205768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:49.207410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:49.319745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:49.321094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cce6ee-1b54-40b4-abe6-977aaf074d2f service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:49.339523 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 733/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:51.557819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4974de1-8b5d-4d3b-95ce-a519c12ecd26 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:51.558806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4974de1-8b5d-4d3b-95ce-a519c12ecd26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:51.558806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4974de1-8b5d-4d3b-95ce-a519c12ecd26 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:51.560707 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:51.568388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d613d30-544b-4df1-aab0-c70a8676e9ef None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:51.568851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d613d30-544b-4df1-aab0-c70a8676e9ef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:51.568947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d613d30-544b-4df1-aab0-c70a8676e9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:51.639199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2d613d30-544b-4df1-aab0-c70a8676e9ef None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:51.640418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2d613d30-544b-4df1-aab0-c70a8676e9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:51.649083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:51.742171 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:52.021256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4974de1-8b5d-4d3b-95ce-a519c12ecd26 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:52.023496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b4974de1-8b5d-4d3b-95ce-a519c12ecd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:52.143555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4974de1-8b5d-4d3b-95ce-a519c12ecd26 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:52.201330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:52.220105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:52.220630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:52.220925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:52.221190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:52.222955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:52.337826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:52.339308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b87f2f-6200-475e-ab42-a304702bffef service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:52.358655 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 727/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:54.572997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211e743-d06f-4fe0-9bf6-decbf34e7d54 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:54.573942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211e743-d06f-4fe0-9bf6-decbf34e7d54 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:54.573942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211e743-d06f-4fe0-9bf6-decbf34e7d54 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:54.575966 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:54.585030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56b86d-68a9-423c-b03f-5105bfd95e4a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:54.585480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56b86d-68a9-423c-b03f-5105bfd95e4a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:54.585585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56b86d-68a9-423c-b03f-5105bfd95e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:54.659986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dd56b86d-68a9-423c-b03f-5105bfd95e4a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:54.661202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dd56b86d-68a9-423c-b03f-5105bfd95e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:54.703193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:54.767265 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:55.052454 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211e743-d06f-4fe0-9bf6-decbf34e7d54 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:55.054012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d211e743-d06f-4fe0-9bf6-decbf34e7d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:55.165518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211e743-d06f-4fe0-9bf6-decbf34e7d54 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:55.206737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:55.244261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:55.244847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:55.245113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:55.245374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:55.246973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:55.360096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:55.361553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65d19e-6f80-462a-9641-7c13e049759c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:55.379733 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 736/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:49:57.574510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf7c3d-e5aa-44ce-b59b-8d82bf8c84ad None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:57.575092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf7c3d-e5aa-44ce-b59b-8d82bf8c84ad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:57.575092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf7c3d-e5aa-44ce-b59b-8d82bf8c84ad None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:57.576438 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:49:57.583144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3de35-b2d0-480d-8ed3-6f0d202a9769 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:57.583999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3de35-b2d0-480d-8ed3-6f0d202a9769 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:57.583999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a3de35-b2d0-480d-8ed3-6f0d202a9769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:57.656615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-78a3de35-b2d0-480d-8ed3-6f0d202a9769 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:49:57.657828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-78a3de35-b2d0-480d-8ed3-6f0d202a9769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:57.676806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:57.759722 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:49:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:49:58.040764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf7c3d-e5aa-44ce-b59b-8d82bf8c84ad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:49:58.043136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2edf7c3d-e5aa-44ce-b59b-8d82bf8c84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:58.168390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf7c3d-e5aa-44ce-b59b-8d82bf8c84ad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:49:58.237006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:49:58.243934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:49:58.244516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:49:58.244785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:49:58.245077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:49:58.246643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:49:58.361783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:49:58.363335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482b45d-fe7d-4be9-b2f1-e8046091fe61 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:49:58.384738 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 730/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:49:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:00.532769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca5e5-0450-4cc6-9151-9f6ddfa6eb78 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:00.533979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca5e5-0450-4cc6-9151-9f6ddfa6eb78 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:00.533979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cca5e5-0450-4cc6-9151-9f6ddfa6eb78 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:00.535207 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:00.543898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408ad6-539a-4473-93cf-0aae3005c1c6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:00.544360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408ad6-539a-4473-93cf-0aae3005c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:00.544463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f408ad6-539a-4473-93cf-0aae3005c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:00.638422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0f408ad6-539a-4473-93cf-0aae3005c1c6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:00.639680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f408ad6-539a-4473-93cf-0aae3005c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:00.703164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:00.750889 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:50:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:01.039403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cca5e5-0450-4cc6-9151-9f6ddfa6eb78 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:01.041867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-38cca5e5-0450-4cc6-9151-9f6ddfa6eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:01.169512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cca5e5-0450-4cc6-9151-9f6ddfa6eb78 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:01.232781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:01.261060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:01.261671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:01.261972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:01.262240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:01.263908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:01.381749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:01.383199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a0458-3b63-4a68-910b-528dbda7d867 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:01.402812 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 739/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:10.749562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=22 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:11.399724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=66 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:21.242232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eda3cf-e05a-474b-96e3-b8f068cd33db None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:21.242820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eda3cf-e05a-474b-96e3-b8f068cd33db None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:21.242820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eda3cf-e05a-474b-96e3-b8f068cd33db None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:21.244139 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 732/1471] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:21.250720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99981a0-3f9a-41e1-9b89-872c43b8d9f7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:21.251498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99981a0-3f9a-41e1-9b89-872c43b8d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:21.251634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99981a0-3f9a-41e1-9b89-872c43b8d9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:21.329985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e99981a0-3f9a-41e1-9b89-872c43b8d9f7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:21.331215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e99981a0-3f9a-41e1-9b89-872c43b8d9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:21.432276 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 740/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:21.732184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22bfa2-f8a3-4824-b271-322e75e6e5b9 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:21.732550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22bfa2-f8a3-4824-b271-322e75e6e5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:21.732872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22bfa2-f8a3-4824-b271-322e75e6e5b9 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:21.734985 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 733/1473] 10.222.0.23 () {54 vars in 966 bytes} [Sun Sep 11 00:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:21.741227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7df37-8efa-4ed7-8dbc-9e5ddacc7f4f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:21.741512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7df37-8efa-4ed7-8dbc-9e5ddacc7f4f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:21.741768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7df37-8efa-4ed7-8dbc-9e5ddacc7f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:21.824963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cfd7df37-8efa-4ed7-8dbc-9e5ddacc7f4f None None] MFA Rules not processed for user `8388bedc588f4f5f94f1e732d46e83b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:21.826183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cfd7df37-8efa-4ed7-8dbc-9e5ddacc7f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:21.847681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:21.910855 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 741/1474] 10.222.0.23 () {58 vars in 1063 bytes} [Sun Sep 11 00:50:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:21.918989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22bfa2-f8a3-4824-b271-322e75e6e5b9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:21.920547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff22bfa2-f8a3-4824-b271-322e75e6e5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:22.029853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22bfa2-f8a3-4824-b271-322e75e6e5b9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:22.105277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:22.105874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:22.106139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:22.106399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:22.108012 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:22.163340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:22.221871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:22.223413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd723e-ad0f-4147-8198-6aa752c52be8 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:22.246258 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 734/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:24.510685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d51c9d-f96f-40fa-a44b-f62d64eebc73 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:24.511611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d51c9d-f96f-40fa-a44b-f62d64eebc73 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:24.511611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d51c9d-f96f-40fa-a44b-f62d64eebc73 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:24.513860 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 742/1476] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:24.520335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32e9b-a5fc-4ccd-a39a-14537008bdc1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:24.521114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32e9b-a5fc-4ccd-a39a-14537008bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:24.521300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b32e9b-a5fc-4ccd-a39a-14537008bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:24.598949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c4b32e9b-a5fc-4ccd-a39a-14537008bdc1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:24.600197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4b32e9b-a5fc-4ccd-a39a-14537008bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:24.708932 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 735/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:24.984695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d51c9d-f96f-40fa-a44b-f62d64eebc73 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:24.986901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-56d51c9d-f96f-40fa-a44b-f62d64eebc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:25.131551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d51c9d-f96f-40fa-a44b-f62d64eebc73 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:25.188928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:25.235924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:25.236444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:25.236713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:25.236974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:25.237635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:25.239401 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:50:25.239401 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:50:25.239401 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:50:25.240534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817ca62-7db7-4dab-af37-8990d4f1206e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:25.259720 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 743/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Sep 11 00:50:24 2022] GET /identity/v3/projects => generated 2553 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:50:27.421024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096f890-2d26-4c6e-962e-201fad97f308 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:27.421890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096f890-2d26-4c6e-962e-201fad97f308 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:27.421890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096f890-2d26-4c6e-962e-201fad97f308 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:27.423944 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 736/1479] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:27.433213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd7ba-7cb9-455a-9389-4818324e7871 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:27.433657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd7ba-7cb9-455a-9389-4818324e7871 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:27.433774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcd7ba-7cb9-455a-9389-4818324e7871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:27.512120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-49fcd7ba-7cb9-455a-9389-4818324e7871 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:27.513330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-49fcd7ba-7cb9-455a-9389-4818324e7871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:27.616476 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 744/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:27.908068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096f890-2d26-4c6e-962e-201fad97f308 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:27.909678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0096f890-2d26-4c6e-962e-201fad97f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:27.920576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:28.063036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096f890-2d26-4c6e-962e-201fad97f308 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:28.153461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:28.190340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:28.190958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:28.191303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:28.191573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:28.193201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:28.297504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:28.299119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be191d43-2269-4415-9a62-bff9df7012d2 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:28.317909 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 737/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:30.645199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef9313-8adf-4ffd-a5c8-622b25c00174 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:30.645831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef9313-8adf-4ffd-a5c8-622b25c00174 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:30.645831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef9313-8adf-4ffd-a5c8-622b25c00174 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:30.647111 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 745/1482] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:30.652727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff92c-4b5d-4e2a-89a0-0a24ade5c66c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:30.653198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff92c-4b5d-4e2a-89a0-0a24ade5c66c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:30.653304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ff92c-4b5d-4e2a-89a0-0a24ade5c66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:30.735215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8b0ff92c-4b5d-4e2a-89a0-0a24ade5c66c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:30.736351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8b0ff92c-4b5d-4e2a-89a0-0a24ade5c66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:30.785881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:30.844138 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 738/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:31.133176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef9313-8adf-4ffd-a5c8-622b25c00174 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:31.134727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c5ef9313-8adf-4ffd-a5c8-622b25c00174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:31.163644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:31.249402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ef9313-8adf-4ffd-a5c8-622b25c00174 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:31.326294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:31.326884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:31.327150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:31.327446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:31.329035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:31.410947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:31.448920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:31.450334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41a40c-e890-4fac-baa5-b0033ea787ef service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:31.468611 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 746/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:50:33.672535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a11681-1b28-4ca3-9b57-dd8c1d3bb1b5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:33.673503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a11681-1b28-4ca3-9b57-dd8c1d3bb1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:33.673503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a11681-1b28-4ca3-9b57-dd8c1d3bb1b5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:33.676278 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 739/1485] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:33.682089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fe01-233e-4b77-83ac-c869aa74b1f3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:33.682513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fe01-233e-4b77-83ac-c869aa74b1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:33.682621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58fe01-233e-4b77-83ac-c869aa74b1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:33.754456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-aa58fe01-233e-4b77-83ac-c869aa74b1f3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:33.755798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aa58fe01-233e-4b77-83ac-c869aa74b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:33.866898 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 747/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:34.177106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a11681-1b28-4ca3-9b57-dd8c1d3bb1b5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:34.178658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e1a11681-1b28-4ca3-9b57-dd8c1d3bb1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:34.293621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a11681-1b28-4ca3-9b57-dd8c1d3bb1b5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:34.368601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:34.370008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:34.370662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:34.370967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:34.371167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:34.373594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:34.509787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:34.511233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9360c-ae32-46bf-9931-cd1bc534724d service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:34.531066 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 740/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:36.740180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc459b-b80d-4d33-b831-2bd5476e33f1 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:36.740743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc459b-b80d-4d33-b831-2bd5476e33f1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:36.740743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc459b-b80d-4d33-b831-2bd5476e33f1 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:36.741961 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 748/1488] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:36.748486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bb771-8d65-4771-9809-0be18df573e0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:36.749247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bb771-8d65-4771-9809-0be18df573e0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:36.749378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0bb771-8d65-4771-9809-0be18df573e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:36.823297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0f0bb771-8d65-4771-9809-0be18df573e0 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:36.824481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f0bb771-8d65-4771-9809-0be18df573e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:36.876961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:36.922678 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 741/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:50:37.215893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc459b-b80d-4d33-b831-2bd5476e33f1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:37.217502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bddc459b-b80d-4d33-b831-2bd5476e33f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:37.254891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:37.326001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc459b-b80d-4d33-b831-2bd5476e33f1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:37.404356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:37.405221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:37.405492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:37.405754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:37.407386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:37.502187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:37.534584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:37.536046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08b624-477e-448a-9f94-e23662972af8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:37.555199 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 749/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:39.956070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0b273-b3a4-49d9-a10c-9677c7e8d2b6 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:39.956707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0b273-b3a4-49d9-a10c-9677c7e8d2b6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:39.956707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0b273-b3a4-49d9-a10c-9677c7e8d2b6 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:39.958141 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 742/1491] 10.222.0.23 () {54 vars in 927 bytes} [Sun Sep 11 00:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:50:39.966545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3088-7d97-4f50-b03b-9203b889a88d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:39.967012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3088-7d97-4f50-b03b-9203b889a88d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:39.967108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3088-7d97-4f50-b03b-9203b889a88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:40.034506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0f6e3088-7d97-4f50-b03b-9203b889a88d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:50:40.035790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0f6e3088-7d97-4f50-b03b-9203b889a88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:40.136084 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 750/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Sep 11 00:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:50:40.428609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0b273-b3a4-49d9-a10c-9677c7e8d2b6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:50:40.430168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-41b0b273-b3a4-49d9-a10c-9677c7e8d2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:40.536399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0b273-b3a4-49d9-a10c-9677c7e8d2b6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:50:40.620001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:50:40.620799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:50:40.621093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:50:40.621380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:50:40.623335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:50:40.635404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:40.740091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:50:40.741398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63d119-7e94-4729-bc91-367f08baabc1 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:50:40.759827 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 743/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:50:50.134261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=82 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:50:50.757597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=50 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:53.657175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:53.657844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:53.657844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:53.737852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:51:53.739068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:53.851031 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 751/1494] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:51:53.862533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:53.863056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:53.863056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:53.944725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:51:53.945854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:54.065784 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 744/1495] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:51:54.073155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:54.074843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:54.089631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b1e255-4451-49a3-b19d-a2dfa9240db7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:54.161320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:54.196639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:54.197163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:54.197431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:54.197692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:54.198324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:51:54.198616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:54.199655 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:54.199655 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:51:54.199655 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:51:54.200547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:54.212765 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 752/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:51:54.221206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:54.230587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbce5265-fa54-475f-9be0-d88c2571ca82 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:54.300902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:54.464238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:54.464977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:54.465421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:54.465858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:54.466954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:54.469839 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:54.469839 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:51:54.469839 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:51:54.472037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:54.531688 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 745/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:51:54 2022] POST /identity/v3/projects => generated 379 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:51:54.536212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:54.538300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:54.574135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd273be3-918f-4536-a92e-7ac695233800 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:54.683074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:54.683849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:54.683968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:54.684226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:54.684852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:54.685859 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:54.685859 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:51:54.685859 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:51:54.687058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:54.736378 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 753/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:51:54 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:51:54.740792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:54.742327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:54.806600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1dd7e2-5b37-4ab0-b0ae-6913dcadf340 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:54.894700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:54.907012 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:54.907644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:54.907949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:54.908211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:54.908849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:54.909865 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:54.909865 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:51:54.909865 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:51:54.910769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:54.923698 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 746/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:51:54.928632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:54.930260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:54.942236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69fc2d-970c-4536-8998-9551ec46b093 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:54.969343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:55.044924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:55.045436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:55.045703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:55.045965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:55.046597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:55.047709 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:55.047709 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:51:55.047709 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:51:55.048580 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f08b245-60a5-4e4f-82dd-d84a30eb8554 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:55.061954 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 754/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:51:55.066505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:55.068183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:55.101756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3954cb49-f98b-4498-8a3a-21a4eb2cb6e5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:55.210931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:55.211880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:55.212176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:55.212492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] PATH_INFO: `/v3/projects/69e14f3a1f064060aa959e7a42c19011/users/d5f6ed7ba8014a1c878d6a6299a8b63d/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:55.238731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e14f3a1f064060aa959e7a42c19011, user_id=d5f6ed7ba8014a1c878d6a6299a8b63d, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:55.240329 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:55.240329 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:51:55.240329 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:51:55.243228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f020ffa-ef63-4388-a562-0f1fb32bbfdc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:55.272853 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 747/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:51:55 2022] PUT /identity/v3/projects/69e14f3a1f064060aa959e7a42c19011/users/d5f6ed7ba8014a1c878d6a6299a8b63d/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:51:55.285296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971c6a1-d0b5-454d-96e5-d5fdc2ba3b7f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:55.285570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971c6a1-d0b5-454d-96e5-d5fdc2ba3b7f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:55.285832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971c6a1-d0b5-454d-96e5-d5fdc2ba3b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:55.347752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1971c6a1-d0b5-454d-96e5-d5fdc2ba3b7f None None] MFA Rules not processed for user `d5f6ed7ba8014a1c878d6a6299a8b63d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:51:55.353510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1971c6a1-d0b5-454d-96e5-d5fdc2ba3b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:55.438488 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 755/1502] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:51:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:51:55.452571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c01e7-408a-4f9e-be4f-714da4c4f64b None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:55.452846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c01e7-408a-4f9e-be4f-714da4c4f64b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:55.453106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c01e7-408a-4f9e-be4f-714da4c4f64b None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:55.454567 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 748/1503] 10.222.0.23 () {54 vars in 964 bytes} [Sun Sep 11 00:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:51:55.461225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2583-a8da-42eb-afe7-7c2c2cd187bf None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:55.461527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2583-a8da-42eb-afe7-7c2c2cd187bf None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:55.461792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2583-a8da-42eb-afe7-7c2c2cd187bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:55.493408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:55.538084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-557c2583-a8da-42eb-afe7-7c2c2cd187bf None None] MFA Rules not processed for user `7d71b946e7d64a50888cfcc337d688a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:51:55.539359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-557c2583-a8da-42eb-afe7-7c2c2cd187bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:55.617621 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 756/1504] 10.222.0.23 () {58 vars in 1061 bytes} [Sun Sep 11 00:51:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:51:55.626243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c01e7-408a-4f9e-be4f-714da4c4f64b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:55.627811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a17c01e7-408a-4f9e-be4f-714da4c4f64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:55.689497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:55.760624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c01e7-408a-4f9e-be4f-714da4c4f64b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:55.836865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:55.837459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:55.837724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:55.837982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:55.839591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:55.943897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:55.945246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:55.948688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:55.964131 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 749/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:51:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:51:56.009943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728f10b-ccd1-4610-9a2f-3c890ae9ed80 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:56.010374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728f10b-ccd1-4610-9a2f-3c890ae9ed80 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:56.010467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728f10b-ccd1-4610-9a2f-3c890ae9ed80 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:56.011843 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 757/1506] 10.222.0.23 () {52 vars in 830 bytes} [Sun Sep 11 00:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:51:56.015773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:56.016967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:56.150221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737c2aab-7312-4dd5-854f-45f9bca5c4e3 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:56.254396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:56.255097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:56.255445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:56.255717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] PATH_INFO: `/v3/users/d5f6ed7ba8014a1c878d6a6299a8b63d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:56.273970 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f6ed7ba8014a1c878d6a6299a8b63d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:56.274998 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:56.274998 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:51:56.274998 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:51:56.276224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:56.286085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:56.349346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f6ed7ba8014a1c878d6a6299a8b63d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:56.360365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f6ed7ba8014a1c878d6a6299a8b63d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:56.385338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f6ed7ba8014a1c878d6a6299a8b63d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:56.385774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f6ed7ba8014a1c878d6a6299a8b63d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:56.400631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f6ed7ba8014a1c878d6a6299a8b63d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:56.406374 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 750/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:51:56 2022] DELETE /identity/v3/users/d5f6ed7ba8014a1c878d6a6299a8b63d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:51:56.412038 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:56.412332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:56.412603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:56.482773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:51:56.484056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:56.562108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:56.575666 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 758/1508] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:51:56.585675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:56.586906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:56.690371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3195bc-98ef-4ed8-aa90-01df70578509 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:56.746845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:51:56.770704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:56.771358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:56.771635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:56.771901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:56.773503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:56.886855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:56.888327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506987-876d-42e8-ae6a-b7f7060652d5 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:56.908907 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 751/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:51:57.360504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:51:57.361757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:51:57.373537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210b678-a5b8-4eb4-877f-e13d95906eb8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:51:57.481451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:51:57.482000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:51:57.482270 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:51:57.482537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] PATH_INFO: `/v3/projects/69e14f3a1f064060aa959e7a42c19011` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:51:57.494829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e14f3a1f064060aa959e7a42c19011)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:51:57.495914 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:51:57.495914 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:51:57.495914 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:51:57.497104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:51:57.561927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e14f3a1f064060aa959e7a42c19011', 'request_id': 'req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:57.576291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e14f3a1f064060aa959e7a42c19011', 'request_id': 'req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:57.590117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e14f3a1f064060aa959e7a42c19011', 'request_id': 'req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:57.612890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] The token cache is being invalidate because project 69e14f3a1f064060aa959e7a42c19011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:51:57.613176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:57.613595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f8da4d26-37a9-4aa2-8c52-4a1dd95adb0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:51:57.615001 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 759/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:51:57 2022] DELETE /identity/v3/projects/69e14f3a1f064060aa959e7a42c19011 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:52:06.906265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=62 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:52:07.610776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:36.452623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2288b-2701-4ea8-ba60-d7ea7ac9bfdd None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:36.453534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2288b-2701-4ea8-ba60-d7ea7ac9bfdd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:36.453534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2288b-2701-4ea8-ba60-d7ea7ac9bfdd None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:36.455699 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:36.460748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e3cb8-acda-4d57-b80d-42b79c3bf6c9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:36.461216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e3cb8-acda-4d57-b80d-42b79c3bf6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:36.461330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e3cb8-acda-4d57-b80d-42b79c3bf6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:36.549399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-536e3cb8-acda-4d57-b80d-42b79c3bf6c9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:36.550614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-536e3cb8-acda-4d57-b80d-42b79c3bf6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:36.660242 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:36.817049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2288b-2701-4ea8-ba60-d7ea7ac9bfdd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:36.820942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-99e2288b-2701-4ea8-ba60-d7ea7ac9bfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:36.958133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2288b-2701-4ea8-ba60-d7ea7ac9bfdd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:36.964310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a03f5-a992-49d9-ade6-882d2501b392 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:36.964922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a03f5-a992-49d9-ade6-882d2501b392 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:36.965198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a03f5-a992-49d9-ade6-882d2501b392 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:36.965468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a03f5-a992-49d9-ade6-882d2501b392 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:36.973955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a03f5-a992-49d9-ade6-882d2501b392 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:36.975565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a03f5-a992-49d9-ade6-882d2501b392 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:36.985121 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 753/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:53:37.492172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7c848-bb86-4ea0-bc98-35d5bfbf3ba2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:37.492730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7c848-bb86-4ea0-bc98-35d5bfbf3ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:37.492730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7c848-bb86-4ea0-bc98-35d5bfbf3ba2 None None] PATH_INFO: `/v3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:37.494020 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 761/1514] 10.222.0.46 () {54 vars in 925 bytes} [Sun Sep 11 00:53:37 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:53:37.528269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b7ba-d599-46c0-8980-5f6353396e5b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:37.528793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b7ba-d599-46c0-8980-5f6353396e5b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:37.528793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b7ba-d599-46c0-8980-5f6353396e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:37.592549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8089b7ba-d599-46c0-8980-5f6353396e5b None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:37.593806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8089b7ba-d599-46c0-8980-5f6353396e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:37.661897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:37.667684 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 754/1515] 10.222.0.46 () {58 vars in 1015 bytes} [Sun Sep 11 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:38.070275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be625767-8aaa-4781-a4f4-58eced067ec4 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:38.070551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be625767-8aaa-4781-a4f4-58eced067ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:38.070799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be625767-8aaa-4781-a4f4-58eced067ec4 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:38.072248 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 762/1516] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:38.079645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09402a2c-909b-4211-92ff-8ee1a637970d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:38.079963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09402a2c-909b-4211-92ff-8ee1a637970d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:38.080184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09402a2c-909b-4211-92ff-8ee1a637970d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:38.109640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-09402a2c-909b-4211-92ff-8ee1a637970d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:38.110849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-09402a2c-909b-4211-92ff-8ee1a637970d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:38.120224 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 755/1517] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:38.274052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be625767-8aaa-4781-a4f4-58eced067ec4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:38.298356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be625767-8aaa-4781-a4f4-58eced067ec4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:38.305644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba7b8ba-1834-47a4-adbe-3ecd1b459ebb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:38.306342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7b8ba-1834-47a4-adbe-3ecd1b459ebb service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:38.306614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7b8ba-1834-47a4-adbe-3ecd1b459ebb service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:38.306878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba7b8ba-1834-47a4-adbe-3ecd1b459ebb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:38.315785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7b8ba-1834-47a4-adbe-3ecd1b459ebb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:38.317375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba7b8ba-1834-47a4-adbe-3ecd1b459ebb service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:38.326483 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 763/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:53:46.367605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb5f6a-7536-4e54-8894-9e9d53467e55 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:46.368107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb5f6a-7536-4e54-8894-9e9d53467e55 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:46.368107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb5f6a-7536-4e54-8894-9e9d53467e55 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:46.369473 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 756/1519] 10.222.0.23 () {54 vars in 961 bytes} [Sun Sep 11 00:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:46.374877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68c844-e173-4f60-956c-0e793f31f164 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:46.375322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68c844-e173-4f60-956c-0e793f31f164 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:46.375516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68c844-e173-4f60-956c-0e793f31f164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:46.401873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4c68c844-e173-4f60-956c-0e793f31f164 None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:46.403119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4c68c844-e173-4f60-956c-0e793f31f164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:46.413483 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 764/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Sep 11 00:53:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:46.421423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb5f6a-7536-4e54-8894-9e9d53467e55 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:46.437464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb5f6a-7536-4e54-8894-9e9d53467e55 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:46.442640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb14508e-cdcc-44d2-a1f8-108205397d2b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:46.443339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14508e-cdcc-44d2-a1f8-108205397d2b service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:46.443610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14508e-cdcc-44d2-a1f8-108205397d2b service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:46.443873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb14508e-cdcc-44d2-a1f8-108205397d2b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:46.451908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14508e-cdcc-44d2-a1f8-108205397d2b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:46.453467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb14508e-cdcc-44d2-a1f8-108205397d2b service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:46.462943 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 757/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:53:49.824169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc3e8f-9642-4aea-9f28-122c2aba6602 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:49.825091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc3e8f-9642-4aea-9f28-122c2aba6602 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:49.825091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc3e8f-9642-4aea-9f28-122c2aba6602 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:49.826805 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:53:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:49.833946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0109f1-669d-4cdb-9024-fdf93182a828 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:49.834386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0109f1-669d-4cdb-9024-fdf93182a828 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:49.834501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0109f1-669d-4cdb-9024-fdf93182a828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:49.861602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0a0109f1-669d-4cdb-9024-fdf93182a828 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:49.862804 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0a0109f1-669d-4cdb-9024-fdf93182a828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:49.870653 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:50.024066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc3e8f-9642-4aea-9f28-122c2aba6602 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:50.040877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc3e8f-9642-4aea-9f28-122c2aba6602 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:50.046405 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beac3d-5089-4b81-891d-7540fa7359ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:50.046991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beac3d-5089-4b81-891d-7540fa7359ab service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:50.047637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beac3d-5089-4b81-891d-7540fa7359ab service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:50.047907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beac3d-5089-4b81-891d-7540fa7359ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:50.056207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51beac3d-5089-4b81-891d-7540fa7359ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:50.057790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51beac3d-5089-4b81-891d-7540fa7359ab service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:50.067683 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 766/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:53:51.121540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8225668-022b-40da-98d4-e9f7ee78457a None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:51.122489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8225668-022b-40da-98d4-e9f7ee78457a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:51.122489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8225668-022b-40da-98d4-e9f7ee78457a None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:51.124632 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Sep 11 00:53:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:51.131267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75425-6365-4093-98b5-bea972a042ec None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:51.131744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75425-6365-4093-98b5-bea972a042ec None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:51.131863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75425-6365-4093-98b5-bea972a042ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:51.154479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dbc75425-6365-4093-98b5-bea972a042ec None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:51.155726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dbc75425-6365-4093-98b5-bea972a042ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:51.166572 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Sep 11 00:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:51.317786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8225668-022b-40da-98d4-e9f7ee78457a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:51.337966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8225668-022b-40da-98d4-e9f7ee78457a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:51.345159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61738a84-06d2-4337-9a5f-486b5c652835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:51.345755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61738a84-06d2-4337-9a5f-486b5c652835 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:51.346024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61738a84-06d2-4337-9a5f-486b5c652835 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:51.346280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61738a84-06d2-4337-9a5f-486b5c652835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:51.355457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61738a84-06d2-4337-9a5f-486b5c652835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:51.357172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61738a84-06d2-4337-9a5f-486b5c652835 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:51.367059 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 760/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:53:55.506208 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be44895-7670-40b5-8038-31c348893c62 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:55.506827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be44895-7670-40b5-8038-31c348893c62 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:55.506827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be44895-7670-40b5-8038-31c348893c62 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:55.508057 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:55.513593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dc30d-6534-4048-a5d1-244e11879042 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:55.514033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dc30d-6534-4048-a5d1-244e11879042 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:55.514135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dc30d-6534-4048-a5d1-244e11879042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:55.536393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-905dc30d-6534-4048-a5d1-244e11879042 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:55.537599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-905dc30d-6534-4048-a5d1-244e11879042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:55.546060 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:55.843072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be44895-7670-40b5-8038-31c348893c62 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:55.845198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4be44895-7670-40b5-8038-31c348893c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:55.885850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:53:55.936274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be44895-7670-40b5-8038-31c348893c62 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:55.941316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2f94f-5200-476c-9a44-13e28bb4b44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:55.941921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2f94f-5200-476c-9a44-13e28bb4b44c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:55.942187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2f94f-5200-476c-9a44-13e28bb4b44c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:55.942446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2f94f-5200-476c-9a44-13e28bb4b44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:55.952801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2f94f-5200-476c-9a44-13e28bb4b44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:55.954239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd2f94f-5200-476c-9a44-13e28bb4b44c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:55.964134 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 769/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:53:58.082225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae4547-2794-4330-8ffe-c15c6c1a2e15 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:58.083168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae4547-2794-4330-8ffe-c15c6c1a2e15 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:58.083168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eae4547-2794-4330-8ffe-c15c6c1a2e15 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:58.085423 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:53:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:53:58.090837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de67b-b531-4e77-af10-cfbe034db03e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:58.091321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de67b-b531-4e77-af10-cfbe034db03e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:58.091425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de67b-b531-4e77-af10-cfbe034db03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:58.112909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5a6de67b-b531-4e77-af10-cfbe034db03e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:53:58.114133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a6de67b-b531-4e77-af10-cfbe034db03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:53:58.123704 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:53:58.413737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae4547-2794-4330-8ffe-c15c6c1a2e15 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:53:58.434453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eae4547-2794-4330-8ffe-c15c6c1a2e15 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:53:58.439448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d009c-4eaa-48f4-b3b3-1555fbc500e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:53:58.440050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d009c-4eaa-48f4-b3b3-1555fbc500e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:53:58.440328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d009c-4eaa-48f4-b3b3-1555fbc500e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:53:58.440604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d009c-4eaa-48f4-b3b3-1555fbc500e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:53:58.450550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d009c-4eaa-48f4-b3b3-1555fbc500e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:53:58.452017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d009c-4eaa-48f4-b3b3-1555fbc500e4 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:53:58.461453 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 763/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:54:02.053838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436e5e-fe38-4b68-81dd-eed50822b8e2 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:54:02.054704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436e5e-fe38-4b68-81dd-eed50822b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:54:02.054704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436e5e-fe38-4b68-81dd-eed50822b8e2 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:54:02.056918 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:54:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:54:02.063996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e02ea-7042-4998-ae85-c0ee9a4a56f9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:54:02.064430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e02ea-7042-4998-ae85-c0ee9a4a56f9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:54:02.064531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27e02ea-7042-4998-ae85-c0ee9a4a56f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:54:02.086234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f27e02ea-7042-4998-ae85-c0ee9a4a56f9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:54:02.087441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f27e02ea-7042-4998-ae85-c0ee9a4a56f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:54:02.097839 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:54:02.395994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b436e5e-fe38-4b68-81dd-eed50822b8e2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:54:02.397910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7b436e5e-fe38-4b68-81dd-eed50822b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:54:02.492625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b436e5e-fe38-4b68-81dd-eed50822b8e2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:54:02.496422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045031e-25f7-411e-9008-29f87cbe959f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:54:02.497022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045031e-25f7-411e-9008-29f87cbe959f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:54:02.497291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045031e-25f7-411e-9008-29f87cbe959f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:54:02.497583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045031e-25f7-411e-9008-29f87cbe959f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:54:02.506623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045031e-25f7-411e-9008-29f87cbe959f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:54:02.508096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045031e-25f7-411e-9008-29f87cbe959f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:54:02.517409 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 772/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:54:04.656143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc10c7-419c-42c2-852d-85d64cf19438 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:54:04.656751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc10c7-419c-42c2-852d-85d64cf19438 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:54:04.656751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc10c7-419c-42c2-852d-85d64cf19438 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:54:04.658041 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Sep 11 00:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:54:04.664368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9f597-c4b9-463a-80ef-3dae7ba384be None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:54:04.664822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9f597-c4b9-463a-80ef-3dae7ba384be None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:54:04.664934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9f597-c4b9-463a-80ef-3dae7ba384be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:54:04.689280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-92a9f597-c4b9-463a-80ef-3dae7ba384be None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:54:04.690492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-92a9f597-c4b9-463a-80ef-3dae7ba384be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:54:04.699817 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Sep 11 00:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:54:04.999321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc10c7-419c-42c2-852d-85d64cf19438 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:54:05.001426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5ffc10c7-419c-42c2-852d-85d64cf19438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:54:05.019517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc10c7-419c-42c2-852d-85d64cf19438 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:54:05.026155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e71da-6924-4d10-99c0-aea46e2f27fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:54:05.027000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e71da-6924-4d10-99c0-aea46e2f27fd service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:54:05.027259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e71da-6924-4d10-99c0-aea46e2f27fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:54:05.027539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e71da-6924-4d10-99c0-aea46e2f27fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:54:05.037864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e71da-6924-4d10-99c0-aea46e2f27fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:54:05.039303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e71da-6924-4d10-99c0-aea46e2f27fd service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:54:05.049025 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 766/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:54:14.683552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=63 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:54:15.044744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=37 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:32.843363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e83dc9-60c8-424c-bd62-f75264545f81 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:32.844072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e83dc9-60c8-424c-bd62-f75264545f81 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:32.844072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e83dc9-60c8-424c-bd62-f75264545f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:32.871211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acddf-ef77-439c-bfba-e2de4e561030 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:32.872317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acddf-ef77-439c-bfba-e2de4e561030 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:32.873001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51acddf-ef77-439c-bfba-e2de4e561030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:32.876425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c2e83dc9-60c8-424c-bd62-f75264545f81 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:32.877938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c2e83dc9-60c8-424c-bd62-f75264545f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:32.895886 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 774/1540] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:32.905048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:32.905360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:32.905604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:32.906747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c51acddf-ef77-439c-bfba-e2de4e561030 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:32.908385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c51acddf-ef77-439c-bfba-e2de4e561030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:32.922488 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 767/1541] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:32.931235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:32.931839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:32.932993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:32.948231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:32.949456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:32.964519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:32.966072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:32.972656 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 775/1542] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:32.977389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:32.977667 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 768/1543] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:32.981710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:32.995740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b125d9-ec32-459f-a250-967ed1e86d26 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:32.997793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d968f0-412b-4f69-bbb9-cdc2a4bd4a28 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.005555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.006485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.006976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.007483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.008530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.009051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.010293 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:55:33.010293 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:55:33.010293 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:55:33.011596 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30722e-392d-4df1-b580-14b93060bb06 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.015860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.016493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.016772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.017040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.017771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.018076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.019717 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.032772 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 769/1544] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:55:33.035562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.036290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.037617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.044486 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 776/1545] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:55:33.044486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.063225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d60e2a-571f-406c-a85a-224b6487d12b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.081435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.082691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.092133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.092765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.093041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.093313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.093966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.094997 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:55:33.094997 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:55:33.094997 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:55:33.096275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5c6a4-3b04-4fe7-a372-7bdb0a75023a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.123138 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 770/1546] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.124009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.136098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0f84-90a6-4e34-bc69-595e3c1506ed None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.160933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9800060-81e4-459e-ba1c-6356cc3275e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.160933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800060-81e4-459e-ba1c-6356cc3275e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.160933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800060-81e4-459e-ba1c-6356cc3275e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.160933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9800060-81e4-459e-ba1c-6356cc3275e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.161574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800060-81e4-459e-ba1c-6356cc3275e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.162175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9800060-81e4-459e-ba1c-6356cc3275e2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.190778 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 777/1547] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.192567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc786c8-4182-4940-b9aa-e2827bcb2a7f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.193042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc786c8-4182-4940-b9aa-e2827bcb2a7f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.193370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc786c8-4182-4940-b9aa-e2827bcb2a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.206020 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 771/1548] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.207314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff96a59-8d02-40fd-af81-e9d3f7b05a76 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.207598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff96a59-8d02-40fd-af81-e9d3f7b05a76 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.207868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff96a59-8d02-40fd-af81-e9d3f7b05a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.214802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cbc786c8-4182-4940-b9aa-e2827bcb2a7f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.216695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cbc786c8-4182-4940-b9aa-e2827bcb2a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.226791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4ff96a59-8d02-40fd-af81-e9d3f7b05a76 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.228013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ff96a59-8d02-40fd-af81-e9d3f7b05a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.278751 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 778/1549] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.280027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72274dd-bc5e-4842-b40f-acdc41ac18a3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.280309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72274dd-bc5e-4842-b40f-acdc41ac18a3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.280580 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72274dd-bc5e-4842-b40f-acdc41ac18a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.290107 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 772/1550] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.290107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e11c8-e892-4926-bdf4-abe67ec7f44e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.290107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e11c8-e892-4926-bdf4-abe67ec7f44e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.290107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e11c8-e892-4926-bdf4-abe67ec7f44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.301458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b72274dd-bc5e-4842-b40f-acdc41ac18a3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.302648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b72274dd-bc5e-4842-b40f-acdc41ac18a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.310421 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 779/1551] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.311625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.311883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.312153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.314814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-274e11c8-e892-4926-bdf4-abe67ec7f44e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.316037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-274e11c8-e892-4926-bdf4-abe67ec7f44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.324010 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 773/1552] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.325186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.325477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.326056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.331095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.332397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.340566 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 780/1553] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.341292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.345171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.346431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.354905 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 774/1554] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.355716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.358429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ff919-a539-4c20-8754-d14192b8f5ff None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.365089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e076aa-0ce2-4150-86a3-688e490dc98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.365651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c76d94-04a6-416d-bcd9-62c664fffeb9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.365760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e076aa-0ce2-4150-86a3-688e490dc98e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.365917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e076aa-0ce2-4150-86a3-688e490dc98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.366196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e076aa-0ce2-4150-86a3-688e490dc98e admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.366828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e076aa-0ce2-4150-86a3-688e490dc98e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.368792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e076aa-0ce2-4150-86a3-688e490dc98e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.370338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.370959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.371333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.371602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.372222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.373237 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:55:33.373237 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:55:33.373237 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:55:33.374430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5658e77-5fb4-4abe-b123-130e69386409 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.393049 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 781/1555] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/users => generated 294 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.394278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.394583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.394818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.397782 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 775/1556] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.398819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090d24-e0af-4aa5-8266-e75f1a95ff92 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.399095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090d24-e0af-4aa5-8266-e75f1a95ff92 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.399401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8090d24-e0af-4aa5-8266-e75f1a95ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.414900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.416161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.418864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c8090d24-e0af-4aa5-8266-e75f1a95ff92 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.420022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8090d24-e0af-4aa5-8266-e75f1a95ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.424702 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 782/1557] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.428463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.428693 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 776/1558] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.429879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54b004-4aa3-4af9-8aac-98624270603c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.430150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54b004-4aa3-4af9-8aac-98624270603c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.430415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d54b004-4aa3-4af9-8aac-98624270603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.442942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb5d1f-774d-455a-a4a9-9a87ad4cfdac None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.449768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7d54b004-4aa3-4af9-8aac-98624270603c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.450445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.450963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7d54b004-4aa3-4af9-8aac-98624270603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.451480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.451943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.452406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.453546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.454075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.456017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33cf30-5361-40b7-83f4-d5e0c7a138dd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.463068 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 777/1559] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.464436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.464733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.465175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.478779 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 783/1560] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:33.478779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.478779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.478779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.486084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.487359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.495213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:33.496407 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 778/1561] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.496563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:33.497193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.507215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92c498-d608-47ff-b852-018ae4655b89 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.507858 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 784/1562] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:33.508696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.515593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.516196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.516465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.516728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.517361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.518258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64ce8f-eb0f-4f71-9002-981b2eceff22 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.518950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.523187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.523843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.524129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.524388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.525027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.526621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.529010 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 779/1563] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:33.533039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.537468 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 785/1564] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:33.538311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.544141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674de8f0-1025-41ae-95db-2ea67379fa0f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.548488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8cd02e-e355-4f3e-8fd0-7d54ddbc4817 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.549152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.549746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.550013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.550278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.550951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.551276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.552849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.553080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.553661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.553931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.554202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.554827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.555125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.556731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.564792 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 780/1565] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:33.565569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.569078 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 786/1566] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:33.569904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.581210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10141556-b5f6-4560-ab22-02a3a29e55cf admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.581396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0db9898-e67a-4a76-b216-b8a11b568fc6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.585890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.586254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.586479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.586746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.587037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.587140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.587450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.587757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.587865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.588051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.588586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.589642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.590584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.601110 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 781/1567] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:33.601886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.614013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ad9c8-27d5-4f7d-9233-81193ea82e6c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.615610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:33.618790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.619418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.619686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.619950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.620634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.620868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.622673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.622906 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 787/1568] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.626892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.635010 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 782/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:33.635695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.641608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:33.646386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63b7b5-34e7-4c56-b28d-259bbc9f95cc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.695834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7de13-860e-41cb-b748-084c3e6d6576 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.701014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.701615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.701885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.702149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.702669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.702891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:33.703211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.703332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.703603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.703873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.704511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.704809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.706167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.716207 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 783/1570] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:33.716720 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 788/1571] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:33.717333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.720049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.727404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e2fc3-4d65-499a-896f-2515ed1f9c8c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.730999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984c3382-dce4-4e16-a84a-a4a51f90ad2d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.731938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.732521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.732905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.733180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.733804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.735722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.736267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.736974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.737250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.737516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.738148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.739805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.750563 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 784/1572] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:33.751373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.761903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdfca9-806a-43ad-ab11-ad60445c189b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.767245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.767772 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 789/1573] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.768014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.768357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.768581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.768675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.769242 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.771414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.802438 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 785/1574] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.803474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.831257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c8c4a-2fed-4234-84b4-8ab9c0558743 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.866572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b153c9d-b806-461a-80d2-404211199a28 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.871967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.872623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.872846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.873110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.873735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.875698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.886442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.887064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.887381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.887678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.888343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.890469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.901755 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 786/1575] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.901755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.909539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87e6973-672b-4c2e-b202-176ed04c34ed admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.914405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:33.914999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:33.915300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:33.915567 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:33.916187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:33.918065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:33.922315 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 790/1576] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.922315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.925287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:33.951268 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 787/1577] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:33.951268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:33.983400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa830d5-9529-4f1f-b5c0-1e06c3d0328d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:33.988523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:34.017753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440f7d-424c-41e8-9c3a-df32c745ec13 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.022339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.023000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.023308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.023580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.024213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.026150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.048179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.048874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.049149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.049419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] PATH_INFO: `/v3/projects/1bf8c10e733e485f9ac63769727b7151/users/44c365d540894a908b34fe51d854e96e/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.053848 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 788/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:33 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:34.057864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.064003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf8c10e733e485f9ac63769727b7151, user_id=44c365d540894a908b34fe51d854e96e, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.065179 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:55:34.065179 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:55:34.065179 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:55:34.067112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.080353 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 791/1579] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:33 2022] PUT /identity/v3/projects/1bf8c10e733e485f9ac63769727b7151/users/44c365d540894a908b34fe51d854e96e/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:34.081204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.121877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1694e11-44a0-4066-b2ce-53372ebda0b4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.141908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91394983-de0b-4acc-98cb-70fd234b5335 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.146728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.147766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.151425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.151713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.152414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.154563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.178018 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 792/1580] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:34.178892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.182656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.183381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.183661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.183930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] PATH_INFO: `/v3/projects/9592f4e25e1e456fa2aecdf1e14b5af3/users/9ec70451a9e04c2fb6dbb7d2f49e1b20/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.189138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fc7c5-1247-423a-a695-6bbf40d37a72 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.193832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.194417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.194690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.194954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.195608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.197749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.199399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9592f4e25e1e456fa2aecdf1e14b5af3, user_id=9ec70451a9e04c2fb6dbb7d2f49e1b20, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.202075 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.217559 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 789/1581] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:34 2022] PUT /identity/v3/projects/9592f4e25e1e456fa2aecdf1e14b5af3/users/9ec70451a9e04c2fb6dbb7d2f49e1b20/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:34.218581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.222402 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 793/1582] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/users => generated 285 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:34.222402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.232090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e27a8-535e-4c46-865e-f2a89120873d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.235811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:34.279465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:34.289217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.289872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.290146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.290411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.291039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.291351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58c136-df94-4f4b-b5f6-b451b6cdfd20 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.292997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.296763 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.297376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.297653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.297921 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.298575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.300219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.311164 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 790/1583] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.311809 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.321326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86699183-616d-482d-aed0-d24c211cfee1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.322328 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 794/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:34.323152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.325984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa541a-f16d-4f94-b30c-7275ccf870ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.326566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa541a-f16d-4f94-b30c-7275ccf870ce admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.326845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa541a-f16d-4f94-b30c-7275ccf870ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.327114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa541a-f16d-4f94-b30c-7275ccf870ce admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.327760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa541a-f16d-4f94-b30c-7275ccf870ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.329650 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa541a-f16d-4f94-b30c-7275ccf870ce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.333027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e6569-e3e9-40e7-a675-878926b4bdbf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.337993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.338695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.338974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.339287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.339918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.342044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.351281 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 791/1585] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:34.352529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.352810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.353135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.366229 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 795/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:34.367114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.376454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:34.377305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17497945-50f9-4394-ab8b-6b9286ac25b1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.377726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:34.382230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44b956-7563-474a-bdf5-0c42c5fe27c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.382819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b956-7563-474a-bdf5-0c42c5fe27c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.383094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b956-7563-474a-bdf5-0c42c5fe27c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.383402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44b956-7563-474a-bdf5-0c42c5fe27c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.384030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44b956-7563-474a-bdf5-0c42c5fe27c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.385326 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 792/1587] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:34.385616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44b956-7563-474a-bdf5-0c42c5fe27c1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.386087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.395506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48fc4a-7f51-43d6-be50-f08bac32430c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.398651 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 796/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.399901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.400306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.400596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.401322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.402106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.402579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.402933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.403640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.405221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.418829 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 793/1589] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.418829 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.426718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cf073-4455-4d75-8bee-7ca98cf8eaed admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.429353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] MFA Rules not processed for user `9ec70451a9e04c2fb6dbb7d2f49e1b20`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:34.430573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:34.431472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.432058 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.432330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.432594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.433234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.434810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.447982 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 794/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.447982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.455610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733a50-2526-4cc5-bce9-20c439319ef3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.460693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.461281 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.461556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.461823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.462453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.464066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.477466 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 795/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.477466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.487591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5093a-a50b-4ff4-90a4-3b19115b0b72 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.492240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.492785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.493107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.493347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.493964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.495378 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 797/1592] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:34.495586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.496242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.508932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70121bbf-81e7-4c74-9301-6c7e13a4a478 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.517760 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 796/1593] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.517760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.518067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.518067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.518067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.518067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.518458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.519531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.520737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:34.532168 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 798/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.532714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.547673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239ead4-4ac9-46fc-8f79-475cc1fd00c8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.554584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.555489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.555897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.556207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.556866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.559359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.577040 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 799/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.577040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.584719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78e7f4-1ea1-4fda-bed5-1e4f3656ef14 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.589298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.589879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.590014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcac70-d701-449c-9bd2-de5e67ef7f6d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.590154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.590418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.591050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.592672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.593579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.594194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.594471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.594744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.606482 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 800/1596] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.606688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.606803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.606897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.613794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84a930-421c-441d-9032-1dd9f7be8f0f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.615763 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 797/1597] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:34.616604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.618563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca2a23-69d2-4384-a34a-ea2fbdcaaff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.619262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2a23-69d2-4384-a34a-ea2fbdcaaff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.619539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2a23-69d2-4384-a34a-ea2fbdcaaff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.619807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca2a23-69d2-4384-a34a-ea2fbdcaaff0 admin admin] PATH_INFO: `/v3/projects/ac8c0850e1ac43e7913a88dbb7b7e9f8/users/03a3c989ecef42e084fb44d47c55b27e/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.627554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2a1e7-f417-4f2e-b640-d501edbbcd4c service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.632667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.633276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.633553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.633821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.634464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.635690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2a23-69d2-4384-a34a-ea2fbdcaaff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8c0850e1ac43e7913a88dbb7b7e9f8, user_id=03a3c989ecef42e084fb44d47c55b27e, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.636095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.638578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca2a23-69d2-4384-a34a-ea2fbdcaaff0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.651577 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 798/1598] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.651577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.654557 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 801/1599] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:34 2022] PUT /identity/v3/projects/ac8c0850e1ac43e7913a88dbb7b7e9f8/users/03a3c989ecef42e084fb44d47c55b27e/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:34.654847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:34.656886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb76d1c-160b-475c-bc12-e96a732259a3 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.657317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb76d1c-160b-475c-bc12-e96a732259a3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.657626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb76d1c-160b-475c-bc12-e96a732259a3 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.659069 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 802/1600] 10.222.0.23 () {54 vars in 962 bytes} [Sun Sep 11 00:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:34.660052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb76d1c-160b-475c-bc12-e96a732259a3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.664170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2b47d-52a2-4178-a028-7451f6c43c77 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.729159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.730196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.730654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.730813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.731476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.733308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.737779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb76d1c-160b-475c-bc12-e96a732259a3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.743443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.744149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.744427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.744695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.748311 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 799/1601] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.748476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.748564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.748640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.758080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c6d89-6208-4f10-9c60-a49cf0941199 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.758838 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 803/1602] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:34.759814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.765501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61347597-4131-46cb-9982-42f01434496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.766397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61347597-4131-46cb-9982-42f01434496b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.766771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61347597-4131-46cb-9982-42f01434496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.767395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61347597-4131-46cb-9982-42f01434496b admin admin] PATH_INFO: `/v3/projects/89e6e6a587374d53b8515f264dfd8cb6/users/364cf4a7a3bf4e70973c78a5d690e998/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.770248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df083e-418e-4e8f-94d6-4d04eb6194b2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.770385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:34.776891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98b63f-f9de-40eb-93fe-a05f67c52485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.777685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98b63f-f9de-40eb-93fe-a05f67c52485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.778020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98b63f-f9de-40eb-93fe-a05f67c52485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.778311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98b63f-f9de-40eb-93fe-a05f67c52485 admin admin] PATH_INFO: `/v3/projects/03b8ca37caa840dda244ca042e146251/users/02d08e3862b8472fa6f5ae8858862ae7/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.789191 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61347597-4131-46cb-9982-42f01434496b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89e6e6a587374d53b8515f264dfd8cb6, user_id=364cf4a7a3bf4e70973c78a5d690e998, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.792084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61347597-4131-46cb-9982-42f01434496b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.795235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98b63f-f9de-40eb-93fe-a05f67c52485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b8ca37caa840dda244ca042e146251, user_id=02d08e3862b8472fa6f5ae8858862ae7, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:34.798477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98b63f-f9de-40eb-93fe-a05f67c52485 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:34.812463 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 800/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:34 2022] PUT /identity/v3/projects/89e6e6a587374d53b8515f264dfd8cb6/users/364cf4a7a3bf4e70973c78a5d690e998/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:34.813230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61347597-4131-46cb-9982-42f01434496b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:34.822499 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 804/1604] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:34 2022] PUT /identity/v3/projects/03b8ca37caa840dda244ca042e146251/users/02d08e3862b8472fa6f5ae8858862ae7/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:34.824023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b29ae3-f2f8-4b54-a447-65e2b29c7b15 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.824365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b29ae3-f2f8-4b54-a447-65e2b29c7b15 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.824564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b29ae3-f2f8-4b54-a447-65e2b29c7b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.849395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-51b29ae3-f2f8-4b54-a447-65e2b29c7b15 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:34.851515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-51b29ae3-f2f8-4b54-a447-65e2b29c7b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:34.901094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61347597-4131-46cb-9982-42f01434496b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:34.924662 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 805/1605] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:34.926821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.927163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.927464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:34.974635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:34.976002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:34.976617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:34.977360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:34.977662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:34.977866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] PATH_INFO: `/v3/projects/4dec451848364c2787673580ec4c90c1/users/081d365c7b15443aa76c7da2ca444a8e/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.016326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dec451848364c2787673580ec4c90c1, user_id=081d365c7b15443aa76c7da2ca444a8e, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.019553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.040024 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 801/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:34 2022] PUT /identity/v3/projects/4dec451848364c2787673580ec4c90c1/users/081d365c7b15443aa76c7da2ca444a8e/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:35.042326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.055597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:35.079052 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 806/1607] 10.222.0.23 () {58 vars in 1059 bytes} [Sun Sep 11 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:35.079913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.092784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec612f1-64d8-41e2-b236-b333086d38a0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.125578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9d7fa8-8151-4d8a-ae51-114a96eb3b18 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.130814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a914b-e7e4-4042-9126-0c0a80414d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.131789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a914b-e7e4-4042-9126-0c0a80414d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.132053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a914b-e7e4-4042-9126-0c0a80414d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.132430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a914b-e7e4-4042-9126-0c0a80414d59 admin admin] PATH_INFO: `/v3/projects/a2d58c81f2e94dd79d2fdcf08b260636/users/f06b14789a2740eca22001c0b5232fc4/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.154513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a914b-e7e4-4042-9126-0c0a80414d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d58c81f2e94dd79d2fdcf08b260636, user_id=f06b14789a2740eca22001c0b5232fc4, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.159944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a914b-e7e4-4042-9126-0c0a80414d59 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.164692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:35.168060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d2b3af-b8a7-4e94-a781-9a02d2157e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.168949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2b3af-b8a7-4e94-a781-9a02d2157e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.169099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2b3af-b8a7-4e94-a781-9a02d2157e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.169250 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d2b3af-b8a7-4e94-a781-9a02d2157e26 admin admin] PATH_INFO: `/v3/projects/63444830fe5d444ab62aeb91347eb229/users/eb838183737c40728ed9e105d17461ea/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.178834 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 802/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:35 2022] PUT /identity/v3/projects/a2d58c81f2e94dd79d2fdcf08b260636/users/f06b14789a2740eca22001c0b5232fc4/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:35.180956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.181604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.181983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.189624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2b3af-b8a7-4e94-a781-9a02d2157e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63444830fe5d444ab62aeb91347eb229, user_id=eb838183737c40728ed9e105d17461ea, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.192554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d2b3af-b8a7-4e94-a781-9a02d2157e26 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.202378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:35.204012 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:35.215267 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 807/1609] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:35 2022] PUT /identity/v3/projects/63444830fe5d444ab62aeb91347eb229/users/eb838183737c40728ed9e105d17461ea/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:35.217686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ab91-5572-4e9a-b1fe-1543bfd80c80 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.218520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ab91-5572-4e9a-b1fe-1543bfd80c80 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.218953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025ab91-5572-4e9a-b1fe-1543bfd80c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.252892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2025ab91-5572-4e9a-b1fe-1543bfd80c80 None None] MFA Rules not processed for user `02d08e3862b8472fa6f5ae8858862ae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:35.254403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2025ab91-5572-4e9a-b1fe-1543bfd80c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:35.278401 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 803/1610] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:35.279601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.326708 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 808/1611] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:35.331768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.332092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.332374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.354598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:35.356103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:35.368156 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 809/1612] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:35.369257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.378805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f2dc72-c7b1-4846-bee9-9f0b1c2190fc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.382499 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbab75-fe3d-44c0-bef2-b95db4846c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.383205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbab75-fe3d-44c0-bef2-b95db4846c5b service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.383629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbab75-fe3d-44c0-bef2-b95db4846c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.383987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbab75-fe3d-44c0-bef2-b95db4846c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.395384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbab75-fe3d-44c0-bef2-b95db4846c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.397353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbab75-fe3d-44c0-bef2-b95db4846c5b service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.408767 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 804/1613] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:35.410356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508099a-8c6e-48d6-b1bc-13ed9b77d145 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.410356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508099a-8c6e-48d6-b1bc-13ed9b77d145 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.411068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5508099a-8c6e-48d6-b1bc-13ed9b77d145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.434639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5508099a-8c6e-48d6-b1bc-13ed9b77d145 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:35.436093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5508099a-8c6e-48d6-b1bc-13ed9b77d145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:35.446116 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 805/1614] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:35.447659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.448079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.448352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.463453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:35.486553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36524bb5-3e2c-41ee-9e19-16f4c7e4c4d2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.494319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] MFA Rules not processed for user `eb838183737c40728ed9e105d17461ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:35.496392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:35.500156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.500797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.501073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.501345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.581417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.582404 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 806/1615] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:35.583405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.588006 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.596629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f426234-fef2-49b1-87d6-6784798fcc59 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.601164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.601827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.602113 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 810/1616] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:35.602248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.602490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.602945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.614961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.615419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab645e-db11-4fd1-aacb-49508b747329 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.616700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.620305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.622000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.622279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.622579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.623369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.625357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.635120 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 807/1617] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:35.637048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.652876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bd81c-32f7-41d3-a780-34a90812586c service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.658530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.659521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.659879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.660397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.673137 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 811/1618] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:35.675868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:35.676989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.678531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.682183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.698426 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 808/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:35.699925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.712426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a59834d-92a3-48d3-9fcb-66a67a38aaa2 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.809421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9415a-5fc1-4849-824c-c35b5ad63ec1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.811932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.812768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.814370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.815310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.816140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.818977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.819150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.820281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.820621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.821730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.844553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.847438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.862768 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 812/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:35.863599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.873415 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 809/1621] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:35.877867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:35.880036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611bc9f2-273e-4a72-bdba-df3bffd57fdb service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.972710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2027a2c-8fef-4395-b8bb-784b94d4f2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.973391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2027a2c-8fef-4395-b8bb-784b94d4f2da admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.973736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2027a2c-8fef-4395-b8bb-784b94d4f2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.974072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2027a2c-8fef-4395-b8bb-784b94d4f2da admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.974986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2027a2c-8fef-4395-b8bb-784b94d4f2da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.977424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2027a2c-8fef-4395-b8bb-784b94d4f2da admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:35.985570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f81af6c-969f-4653-8705-d033d28bad72 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:35.992244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05861706-d07f-4277-905d-4b9695c6a046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:35.993365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05861706-d07f-4277-905d-4b9695c6a046 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:35.993862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05861706-d07f-4277-905d-4b9695c6a046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:35.994428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05861706-d07f-4277-905d-4b9695c6a046 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:35.995724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05861706-d07f-4277-905d-4b9695c6a046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:35.998552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05861706-d07f-4277-905d-4b9695c6a046 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.016379 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 813/1622] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:36.018981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55111-a7eb-4190-88c9-7fc4e1fcb57e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.019380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55111-a7eb-4190-88c9-7fc4e1fcb57e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.020046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55111-a7eb-4190-88c9-7fc4e1fcb57e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.026036 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 814/1623] 10.222.0.23 () {54 vars in 912 bytes} [Sun Sep 11 00:55:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:36.029014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b55111-a7eb-4190-88c9-7fc4e1fcb57e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.034160 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 810/1624] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:35 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:36.035622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.035864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.036136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.045940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:36.057088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b55111-a7eb-4190-88c9-7fc4e1fcb57e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.066141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:36.067682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:36.071406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.072709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.075306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.077158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.079474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.085944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.103508 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 815/1625] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:36.108856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.123996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93998d-443d-48a8-b616-d6b73f4c107b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.130861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.137106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.137457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.145245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.146326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.150036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.175713 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 816/1626] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:36.175713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.187934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e9637-af0f-4d9c-a1bd-d6b0cd56bf2d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.193137 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.194143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.194621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.195126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] PATH_INFO: `/v3/projects/15faea8d8a32488190ca75b43a52f7e0/users/b268cdb50bd64084805bd7eb38e23edd/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.198375 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 811/1627] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:36.199479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.220980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15faea8d8a32488190ca75b43a52f7e0, user_id=b268cdb50bd64084805bd7eb38e23edd, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.221241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:36.225326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.234367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6a4ac-20a1-47ae-9857-84a557c78114 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.245059 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 817/1628] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:36 2022] PUT /identity/v3/projects/15faea8d8a32488190ca75b43a52f7e0/users/b268cdb50bd64084805bd7eb38e23edd/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:36.246373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.321802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c1fec-9a94-4904-b850-387395ffdda3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.326348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.327388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.327762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.328242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.396931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.398652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.400724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.401118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] PATH_INFO: `/v3/projects/913d8a5f819a4fbcae46614659f44363/users/95f52de4d444423eadab4b1086c8817f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.443913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=913d8a5f819a4fbcae46614659f44363, user_id=95f52de4d444423eadab4b1086c8817f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.452651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.480994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.487616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.492516 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 812/1629] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:36 2022] PUT /identity/v3/projects/913d8a5f819a4fbcae46614659f44363/users/95f52de4d444423eadab4b1086c8817f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:36.494157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.536661 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 818/1630] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:36.538189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.577336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a426474-171a-4e2e-8c95-bd300659527e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.650432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66263885-76cc-4fb1-ae91-08054604b332 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.660178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.660971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.661445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.661785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.662788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.664852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.686704 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 813/1631] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:36.691239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.695630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.696699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.697035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.697396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.698249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.700244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.702266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:36.711606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb3a45-9ee7-4b03-b24a-aa79f240c047 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.718272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1210820-2213-49e9-b082-ab265a1f4640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.719224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210820-2213-49e9-b082-ab265a1f4640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.719955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210820-2213-49e9-b082-ab265a1f4640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.720731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210820-2213-49e9-b082-ab265a1f4640 admin admin] PATH_INFO: `/v3/projects/15faea8d8a32488190ca75b43a52f7e0/users/b268cdb50bd64084805bd7eb38e23edd/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.723999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1210820-2213-49e9-b082-ab265a1f4640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15faea8d8a32488190ca75b43a52f7e0, user_id=b268cdb50bd64084805bd7eb38e23edd, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.727808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1210820-2213-49e9-b082-ab265a1f4640 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.734597 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 819/1632] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:36.738839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:36.741561 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 814/1633] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:36 2022] PUT /identity/v3/projects/15faea8d8a32488190ca75b43a52f7e0/users/b268cdb50bd64084805bd7eb38e23edd/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:36.753453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.753873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.754514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.766907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efbb92-56b5-49e2-83ea-1cf0dc55ed87 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:36.849278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] MFA Rules not processed for user `b268cdb50bd64084805bd7eb38e23edd`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:36.849278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:36.928148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab6143-6241-43db-9122-2ed65d5c04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:36.928889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab6143-6241-43db-9122-2ed65d5c04ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.929168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab6143-6241-43db-9122-2ed65d5c04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.929450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab6143-6241-43db-9122-2ed65d5c04ed admin admin] PATH_INFO: `/v3/projects/913d8a5f819a4fbcae46614659f44363/users/95f52de4d444423eadab4b1086c8817f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:36.932353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab6143-6241-43db-9122-2ed65d5c04ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=913d8a5f819a4fbcae46614659f44363, user_id=95f52de4d444423eadab4b1086c8817f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:36.936290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab6143-6241-43db-9122-2ed65d5c04ed admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:36.963015 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 820/1634] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:36 2022] PUT /identity/v3/projects/913d8a5f819a4fbcae46614659f44363/users/95f52de4d444423eadab4b1086c8817f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:55:36.968352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:36.970672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:36.971671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:37.010988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:37.019468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] MFA Rules not processed for user `95f52de4d444423eadab4b1086c8817f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:37.020849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:37.032851 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 815/1635] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:37.042998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:37.221837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d98b2-4cba-4b75-be16-8b87d62f7029 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:37.250798 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 821/1636] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:37.367308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:37.368525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:37.369133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:37.369598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:37.507050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:37.508648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:37.531684 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 816/1637] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:37.622767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:37.636937 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcedb05-1c1f-43c6-94c1-aae031d7d0df None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:37.642712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb159b-cfc5-46ce-aad9-90f7d3686dfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:37.643941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb159b-cfc5-46ce-aad9-90f7d3686dfe service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:37.644279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb159b-cfc5-46ce-aad9-90f7d3686dfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:37.644606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb159b-cfc5-46ce-aad9-90f7d3686dfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:37.660169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb159b-cfc5-46ce-aad9-90f7d3686dfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:37.661784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb159b-cfc5-46ce-aad9-90f7d3686dfe service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:37.691869 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 822/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:37.851501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:37.868863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17c8b20-9934-4153-80fe-fa174d4425b8 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:37.893398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7ab37-f71d-4b67-8fdd-df6fe8293b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:37.894038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ab37-f71d-4b67-8fdd-df6fe8293b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:37.894315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ab37-f71d-4b67-8fdd-df6fe8293b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:37.894582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7ab37-f71d-4b67-8fdd-df6fe8293b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:38.024726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7ab37-f71d-4b67-8fdd-df6fe8293b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:38.026233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7ab37-f71d-4b67-8fdd-df6fe8293b7c service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:38.030016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:38.050956 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 817/1639] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:38.199695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d015e-af63-40fd-a3ac-1a07ccce5e4c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:38.201096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d015e-af63-40fd-a3ac-1a07ccce5e4c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:38.201615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d015e-af63-40fd-a3ac-1a07ccce5e4c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:38.205043 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 823/1640] 10.222.0.23 () {54 vars in 962 bytes} [Sun Sep 11 00:55:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:38.211820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecd506-688a-4858-9033-eb4a1a0d6507 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:38.212221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecd506-688a-4858-9033-eb4a1a0d6507 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:38.212625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecd506-688a-4858-9033-eb4a1a0d6507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:38.246688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2fecd506-688a-4858-9033-eb4a1a0d6507 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:38.248505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2fecd506-688a-4858-9033-eb4a1a0d6507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:38.443235 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 818/1641] 10.222.0.23 () {58 vars in 1059 bytes} [Sun Sep 11 00:55:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:38.450641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca72a3-4bbc-4a72-8b81-d3c9472b213e None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:38.451000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca72a3-4bbc-4a72-8b81-d3c9472b213e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:38.451369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca72a3-4bbc-4a72-8b81-d3c9472b213e None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:38.453061 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 824/1642] 10.222.0.23 () {54 vars in 912 bytes} [Sun Sep 11 00:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:38.460835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f420-5390-447f-a7f8-525005be01ce None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:38.461433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f420-5390-447f-a7f8-525005be01ce None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:38.461848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379f420-5390-447f-a7f8-525005be01ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:38.485449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2379f420-5390-447f-a7f8-525005be01ce None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:38.487305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2379f420-5390-447f-a7f8-525005be01ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:38.508750 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 819/1643] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:55:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:38.526865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca72a3-4bbc-4a72-8b81-d3c9472b213e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:38.549196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca72a3-4bbc-4a72-8b81-d3c9472b213e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:38.565024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:38.567714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:38.568166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:38.569335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:38.603510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:38.608918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:38.666647 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 825/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:40.345373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f420-5390-447f-a7f8-525005be01ce None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:40.515382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379f420-5390-447f-a7f8-525005be01ce None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:40.526890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:40.527957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:40.528477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:40.528856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] PATH_INFO: `/v3/users/9ec70451a9e04c2fb6dbb7d2f49e1b20` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:40.530560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ec70451a9e04c2fb6dbb7d2f49e1b20)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:40.532677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:40.674080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ec70451a9e04c2fb6dbb7d2f49e1b20'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:40.714070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ec70451a9e04c2fb6dbb7d2f49e1b20'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:40.751716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ec70451a9e04c2fb6dbb7d2f49e1b20'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:40.752518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ec70451a9e04c2fb6dbb7d2f49e1b20'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:40.799085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9419a9ee-d0f5-4c04-9a11-731ffbff46b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ec70451a9e04c2fb6dbb7d2f49e1b20'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:40.815906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:40.816491 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 820/1645] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:55:40 2022] DELETE /identity/v3/users/9ec70451a9e04c2fb6dbb7d2f49e1b20 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:40.820836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:40.822292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:40.822675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:40.834123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:40.837033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c7a94-7a2a-4d0a-9fa6-6392a4a2ab1e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:40.858824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:40.862773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:40.933823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:40.960566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:40.963885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:40.964458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:40.964878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] PATH_INFO: `/v3/users/02d08e3862b8472fa6f5ae8858862ae7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:40.968586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02d08e3862b8472fa6f5ae8858862ae7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:40.971357 dv-cinder-master-3-smb devstack@keystone.service[81487]: WARNING py.warnings [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:55:40.971357 dv-cinder-master-3-smb devstack@keystone.service[81487]: warnings.warn(msg) Sep 11 00:55:40.971357 dv-cinder-master-3-smb devstack@keystone.service[81487]: Sep 11 00:55:40.974550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:41.052293 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 821/1646] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:41.074458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02d08e3862b8472fa6f5ae8858862ae7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:41.088949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02d08e3862b8472fa6f5ae8858862ae7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:41.103405 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02d08e3862b8472fa6f5ae8858862ae7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:41.104745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02d08e3862b8472fa6f5ae8858862ae7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:41.129110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c424420c-2682-49dc-8900-c98bb9607262 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02d08e3862b8472fa6f5ae8858862ae7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:41.135407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:41.138338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:41.143622 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 826/1647] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:55:40 2022] DELETE /identity/v3/users/02d08e3862b8472fa6f5ae8858862ae7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:41.144561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:41.144561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:41.144561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:41.180458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:41.180938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:41.280401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee2b9c1-e876-4a5a-8369-22ca9571f1d1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:41.345777 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 827/1648] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:41.376593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:41.378713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:41.470081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:41.472900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:41.474252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:41.474705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:41.481943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:41.561249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:41.566032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:41.611072 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 822/1649] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:41.612088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba421c2-1354-401f-9cd5-70553df6913f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:41.623549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e3fa9-f3df-48fb-bb52-feb9b5a40fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:41.624502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e3fa9-f3df-48fb-bb52-feb9b5a40fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:41.624844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e3fa9-f3df-48fb-bb52-feb9b5a40fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:41.625182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e3fa9-f3df-48fb-bb52-feb9b5a40fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:41.647199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e3fa9-f3df-48fb-bb52-feb9b5a40fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:41.648894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e3fa9-f3df-48fb-bb52-feb9b5a40fd0 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:41.671237 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 828/1650] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:43.431143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:43.434629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:43.458647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e9fcf-02ca-472a-8305-d4b99f1d6778 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:43.478653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:43.480395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:43.480908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:43.481184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] PATH_INFO: `/v3/projects/9592f4e25e1e456fa2aecdf1e14b5af3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:43.482569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9592f4e25e1e456fa2aecdf1e14b5af3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:43.483646 dv-cinder-master-3-smb devstack@keystone.service[81488]: WARNING py.warnings [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 11 00:55:43.483646 dv-cinder-master-3-smb devstack@keystone.service[81488]: warnings.warn(msg) Sep 11 00:55:43.483646 dv-cinder-master-3-smb devstack@keystone.service[81488]: Sep 11 00:55:43.485898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:43.631564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9592f4e25e1e456fa2aecdf1e14b5af3', 'request_id': 'req-e3ba9aec-6209-4fe0-b561-ff46e90b5079'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:43.651688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9592f4e25e1e456fa2aecdf1e14b5af3', 'request_id': 'req-e3ba9aec-6209-4fe0-b561-ff46e90b5079'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:43.654695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:43.669968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9592f4e25e1e456fa2aecdf1e14b5af3', 'request_id': 'req-e3ba9aec-6209-4fe0-b561-ff46e90b5079'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:43.704004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] The token cache is being invalidate because project 9592f4e25e1e456fa2aecdf1e14b5af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:55:43.704198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:43.705533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e3ba9aec-6209-4fe0-b561-ff46e90b5079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:43.707425 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 823/1651] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:55:43 2022] DELETE /identity/v3/projects/9592f4e25e1e456fa2aecdf1e14b5af3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:43.713147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:43.713664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:43.713762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:43.768961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:43.779120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:43.796696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:43.924960 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 829/1652] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:43.932793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:43.933754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:43.934278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:43.979699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:43.984410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:44.026474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.027211 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 824/1653] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:44.044147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5d9eb-c1b7-437e-8b1b-2c0769d16b9d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.067008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:44.077946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:44.077946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:44.077946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:44.077946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:44.077946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:44.080846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:44.113668 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 830/1654] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:44.115609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.133473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce019ecb-e9ac-49a9-87ef-8bf544facc57 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.147262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:44.148250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:44.148712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:44.149188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:44.150039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:44.152414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:44.236551 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 825/1655] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:44 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:44.237810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.252623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f56082-b757-4e61-8ee7-9a2791602001 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.300110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.301931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:44.473570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:44.473570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:44.473570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:44.473570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:44.474774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:44.478865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:44.554014 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 831/1656] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:44 2022] POST /identity/v3/users => generated 303 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:55:44.557398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4890d-d171-44b5-96bb-2652b1f9c388 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.558162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.574070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:44.575553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:44.577095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:44.577700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] PATH_INFO: `/v3/projects/03b8ca37caa840dda244ca042e146251` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:44.589281 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b8ca37caa840dda244ca042e146251)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:44.591811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cee19-731e-4de4-8afe-66076e8ab9f3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.595816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:44.608467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:44.609119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:44.616336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:44.616620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:44.617338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:44.618990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:44.661240 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 832/1657] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:44.669034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.701055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b84a1-a78e-4eaf-911f-060e63df0c43 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.708898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:44.709641 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:44.709941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:44.710506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:44.711261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:44.723778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:44.764586 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 833/1658] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:44.770638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.792358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b8ca37caa840dda244ca042e146251', 'request_id': 'req-335d9dda-fa8e-41d5-b220-78b60ab4793a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:44.810397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab54dd-7de2-4d7a-9b33-484b3daa3fab admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:44.811260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b8ca37caa840dda244ca042e146251', 'request_id': 'req-335d9dda-fa8e-41d5-b220-78b60ab4793a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:44.824381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b8ca37caa840dda244ca042e146251', 'request_id': 'req-335d9dda-fa8e-41d5-b220-78b60ab4793a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:44.866806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] The token cache is being invalidate because project 03b8ca37caa840dda244ca042e146251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:55:44.867222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:44.870700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:44.877206 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 826/1659] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:55:44 2022] DELETE /identity/v3/projects/03b8ca37caa840dda244ca042e146251 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:55:44.878723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:44.885406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:45.001030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e68b70-7ec1-47c5-8a3b-153651af61ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:45.003740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68b70-7ec1-47c5-8a3b-153651af61ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:45.004315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68b70-7ec1-47c5-8a3b-153651af61ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:45.005535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68b70-7ec1-47c5-8a3b-153651af61ef admin admin] PATH_INFO: `/v3/projects/a16210ea6e5f48a3b6ede0b178e6622c/users/5514476e82264e71ae2519fc3145ce1e/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:45.050707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:45.057177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:45.060853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e68b70-7ec1-47c5-8a3b-153651af61ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=a16210ea6e5f48a3b6ede0b178e6622c, user_id=5514476e82264e71ae2519fc3145ce1e, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:45.067936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e68b70-7ec1-47c5-8a3b-153651af61ef admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:45.097221 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 834/1660] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:44 2022] PUT /identity/v3/projects/a16210ea6e5f48a3b6ede0b178e6622c/users/5514476e82264e71ae2519fc3145ce1e/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:45.105464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112353-6477-4435-ac94-169cb42d64dc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:45.105675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112353-6477-4435-ac94-169cb42d64dc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:45.105930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db112353-6477-4435-ac94-169cb42d64dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:45.135758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d9dda-fa8e-41d5-b220-78b60ab4793a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:45.157765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-db112353-6477-4435-ac94-169cb42d64dc None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:45.159406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db112353-6477-4435-ac94-169cb42d64dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:45.284462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:45.285434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:45.285884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:45.286333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] PATH_INFO: `/v3/users/b268cdb50bd64084805bd7eb38e23edd` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:45.288976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b268cdb50bd64084805bd7eb38e23edd)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:45.291165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:45.365064 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 835/1661] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:45.390036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112353-6477-4435-ac94-169cb42d64dc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:45.391743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db112353-6477-4435-ac94-169cb42d64dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:45.430604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b268cdb50bd64084805bd7eb38e23edd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:45.460106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b268cdb50bd64084805bd7eb38e23edd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:45.487798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b268cdb50bd64084805bd7eb38e23edd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:45.489134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b268cdb50bd64084805bd7eb38e23edd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:45.506725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f21e996f-6ece-48df-b39a-6eef7e31322d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b268cdb50bd64084805bd7eb38e23edd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:45.537582 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 827/1662] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:55:44 2022] DELETE /identity/v3/users/b268cdb50bd64084805bd7eb38e23edd => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:45.598055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db112353-6477-4435-ac94-169cb42d64dc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:45.688866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:45.690151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:45.692451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:45.695295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:45.704863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:45.951259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:45.961204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:45.966280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:46.002753 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 836/1663] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:47.165532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:47.165532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:47.166055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:47.242731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] MFA Rules not processed for user `44c365d540894a908b34fe51d854e96e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:47.246142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:47.449305 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 828/1664] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:47.466740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:47.468447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:47.641158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ac871-d213-4ce0-8d2f-836cdaecbe75 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:47.659973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787bc69e-b08c-4a7f-910d-d749c16a5b2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:47.660656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bc69e-b08c-4a7f-910d-d749c16a5b2b service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:47.661619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bc69e-b08c-4a7f-910d-d749c16a5b2b service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:47.661897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787bc69e-b08c-4a7f-910d-d749c16a5b2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:47.677983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:47.681528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:47.698557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:47.706188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787bc69e-b08c-4a7f-910d-d749c16a5b2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:47.711689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0015b009-8f02-4913-a3d8-24dae4fe8785 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:47.712664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787bc69e-b08c-4a7f-910d-d749c16a5b2b service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:47.726106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:47.729263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:47.729504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:47.729776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] PATH_INFO: `/v3/projects/15faea8d8a32488190ca75b43a52f7e0` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:47.735758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=15faea8d8a32488190ca75b43a52f7e0)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:47.745697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:47.752749 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 837/1665] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:55:47.997857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15faea8d8a32488190ca75b43a52f7e0', 'request_id': 'req-46042213-b4b4-4476-83db-20adac963e7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:48.028795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15faea8d8a32488190ca75b43a52f7e0', 'request_id': 'req-46042213-b4b4-4476-83db-20adac963e7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:48.041880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15faea8d8a32488190ca75b43a52f7e0', 'request_id': 'req-46042213-b4b4-4476-83db-20adac963e7e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:48.074452 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] The token cache is being invalidate because project 15faea8d8a32488190ca75b43a52f7e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:55:48.074846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:48.077378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-46042213-b4b4-4476-83db-20adac963e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:48.084252 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 829/1666] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:55:47 2022] DELETE /identity/v3/projects/15faea8d8a32488190ca75b43a52f7e0 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:48.088994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.089575 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.090720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.150335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:48.161441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:48.332365 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 838/1667] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:48.335502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.337359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.341008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.398879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:48.400297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:48.414800 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 830/1668] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:48.415935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:48.426898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4144e72-f567-4822-86f2-a302b01a625d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:48.436561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:48.437425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.437792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.438198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.439040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:55:48.439662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:48.441421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:48.461210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:48.467142 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 839/1669] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:55:48.475493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16c41-aea3-4e08-9905-a5db91fc6ef4 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:48.481915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:48.482791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.483236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.483639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.484409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:48.486618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:48.546116 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 831/1670] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:48 2022] POST /identity/v3/projects => generated 407 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:48.550756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:48.564160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2be9b-a055-41aa-b876-12b5b1e280de admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:48.715835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:48.717025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.717256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.717525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.718304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:48.720517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:48.766116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:48.776935 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 840/1671] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:48 2022] POST /identity/v3/users => generated 304 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:48.778217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:48.797399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c59f0-dcc3-4805-b588-6dba7b87cc96 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:48.812586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:48.814699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.816152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.816616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.817430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:48.827568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:48.850251 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 832/1672] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:48.854268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:48.871906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757c2f8-7583-4909-9673-fa1892ff3026 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:48.880306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf7d2c7-b7fc-44e2-8e5c-60a61db0d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:48.880998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d2c7-b7fc-44e2-8e5c-60a61db0d342 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.881324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d2c7-b7fc-44e2-8e5c-60a61db0d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.881591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7d2c7-b7fc-44e2-8e5c-60a61db0d342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.882335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7d2c7-b7fc-44e2-8e5c-60a61db0d342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:48.885986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf7d2c7-b7fc-44e2-8e5c-60a61db0d342 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:48.903508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:48.903985 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 841/1673] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:48.914871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab191c65-322c-4a59-a27d-702edd13456e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:48.930470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:48.931210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:48.931703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:48.932102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] PATH_INFO: `/v3/projects/2d6af7325f8a486aa88c252d98e1a89e/users/8fd2fe0504674e3cbbb73db14e894eb9/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:48.958009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6af7325f8a486aa88c252d98e1a89e, user_id=8fd2fe0504674e3cbbb73db14e894eb9, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:48.960878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:49.009740 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 833/1674] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:48 2022] PUT /identity/v3/projects/2d6af7325f8a486aa88c252d98e1a89e/users/8fd2fe0504674e3cbbb73db14e894eb9/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:49.020803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:49.023282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:49.023556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:49.109037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] MFA Rules not processed for user `8fd2fe0504674e3cbbb73db14e894eb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:49.115232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:49.344545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:49.346987 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 842/1675] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:49.521626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b56f5e-7124-43e1-ac56-80bb0b53def8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:49.542985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:49.543940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:49.544286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:49.544812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:49.545773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:49.548477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:49.626449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:49.635874 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 834/1676] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:49 2022] POST /identity/v3/projects => generated 409 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:49.647459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b12189-315c-4dfa-8937-355a6c4ef16a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:49.793269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:49.794395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:49.794779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:49.795534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:49.796608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:49.799025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:49.841250 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 843/1677] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:49 2022] POST /identity/v3/users => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:55:49.842968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:49.856053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a39f3b-4715-42ea-beb3-04671db1b1d4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:49.861950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:49.862983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:49.864059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:49.864492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:49.865487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:49.868129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:49.870373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:49.906285 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 835/1678] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:55:49.912068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:49.926075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c538-67c8-45d3-82dd-ce7aa05ad54e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:49.934711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:49.940158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:49.940662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:49.941354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] PATH_INFO: `/v3/projects/e50a893eefa84b7198624e21c7a6e855/users/358d645c2c6244d9bbd2143ba03e1941/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:49.972855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50a893eefa84b7198624e21c7a6e855, user_id=358d645c2c6244d9bbd2143ba03e1941, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:49.975904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:50.009131 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 844/1679] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:49 2022] PUT /identity/v3/projects/e50a893eefa84b7198624e21c7a6e855/users/358d645c2c6244d9bbd2143ba03e1941/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:50.013866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:50.028746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183cb0ef-02d1-4cb3-8057-71948600bd4c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:50.157658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711f906-4c25-4282-afe7-1b9c5c5a51d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:50.158880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711f906-4c25-4282-afe7-1b9c5c5a51d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:50.159211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711f906-4c25-4282-afe7-1b9c5c5a51d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:50.159494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711f906-4c25-4282-afe7-1b9c5c5a51d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:50.160419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711f906-4c25-4282-afe7-1b9c5c5a51d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:50.162517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2711f906-4c25-4282-afe7-1b9c5c5a51d7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:50.177356 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 836/1680] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:50.181819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:50.201179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf9490d-a9fc-46be-88a4-63a19850f952 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:50.208106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:50.208976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:50.209334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:50.209655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] PATH_INFO: `/v3/projects/e50a893eefa84b7198624e21c7a6e855/users/358d645c2c6244d9bbd2143ba03e1941/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:50.211935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50a893eefa84b7198624e21c7a6e855, user_id=358d645c2c6244d9bbd2143ba03e1941, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:50.217243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:50.263330 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 845/1681] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:50 2022] PUT /identity/v3/projects/e50a893eefa84b7198624e21c7a6e855/users/358d645c2c6244d9bbd2143ba03e1941/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:50.265734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:50.266232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:50.266638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:50.334103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] MFA Rules not processed for user `358d645c2c6244d9bbd2143ba03e1941`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:50.336823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:50.552552 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 837/1682] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:50 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:50.561685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:50.566078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:50.635973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:50.695500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3106edc-bf22-4b7a-8307-e67f271e189c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:50.700144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed368e8-6e19-4656-aa07-451aa448db0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:50.700931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed368e8-6e19-4656-aa07-451aa448db0f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:50.701507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed368e8-6e19-4656-aa07-451aa448db0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:50.701867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed368e8-6e19-4656-aa07-451aa448db0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:50.745512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed368e8-6e19-4656-aa07-451aa448db0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:50.747995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed368e8-6e19-4656-aa07-451aa448db0f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:50.781931 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 846/1683] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:50 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:51.240262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:51.255799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7efb415-e3c2-4b0f-a0e0-5919a1f225ec None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:51.282176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:51.282989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:51.283704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:51.284063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:51.312379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:51.312930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:51.313331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:51.397648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] MFA Rules not processed for user `5514476e82264e71ae2519fc3145ce1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:51.398450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:51.465428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:51.466894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:51.502280 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 838/1684] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:51 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:51.606095 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 847/1685] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:51 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:51.611507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:51.614073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:51.662643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:51.787316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a46cd-3233-4aba-ac91-95696ed6cd40 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:51.796450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:51.797138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:51.797417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:51.797682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:51.800199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:51.804504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:51.879408 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 839/1686] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:55:51 2022] POST /identity/v3/projects => generated 405 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:51.883522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:51.902473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd093d7-94b9-416d-ab61-a14f5b1c879b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:52.074042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:52.075947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:52.076440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:52.076902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:52.078281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:52.083758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:52.169582 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 848/1687] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:55:51 2022] POST /identity/v3/users => generated 309 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:55:52.171595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:52.191050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93496-e42b-4c63-a369-e7a4ab2d4f71 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:52.197991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:52.199101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:52.199637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:52.200071 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:52.201197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:52.204004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:52.224640 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 840/1688] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:52.226016 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:52.261532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e098a2c-8582-4d2e-90c5-e38d1e4b3ff2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:52.271277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:52.272328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:52.272991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:52.273533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] PATH_INFO: `/v3/projects/2a05bc43f32744c68619027e93a517c3/users/1c3869a453324112be04a5743a381f59/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:52.293771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a05bc43f32744c68619027e93a517c3, user_id=1c3869a453324112be04a5743a381f59, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:52.298917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:52.327307 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 849/1689] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:52 2022] PUT /identity/v3/projects/2a05bc43f32744c68619027e93a517c3/users/1c3869a453324112be04a5743a381f59/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:52.331708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:52.351129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e52a92-9204-457d-a218-d377c832a40b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:52.514851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e1c5c-6ded-4620-95f9-681f8a8f1148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:52.515460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e1c5c-6ded-4620-95f9-681f8a8f1148 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:52.515776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e1c5c-6ded-4620-95f9-681f8a8f1148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:52.516089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e1c5c-6ded-4620-95f9-681f8a8f1148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:52.516968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e1c5c-6ded-4620-95f9-681f8a8f1148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:52.522136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e1c5c-6ded-4620-95f9-681f8a8f1148 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:52.554944 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 841/1690] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:55:52.556332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:52.576238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230a3bc-bebb-4c33-a82b-672bfe3c3542 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:52.598484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2259d-de1a-4fb6-8a70-9c87a4e4468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:52.601725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2259d-de1a-4fb6-8a70-9c87a4e4468a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:52.603898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2259d-de1a-4fb6-8a70-9c87a4e4468a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:52.603898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2259d-de1a-4fb6-8a70-9c87a4e4468a admin admin] PATH_INFO: `/v3/projects/2a05bc43f32744c68619027e93a517c3/users/1c3869a453324112be04a5743a381f59/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:52.608065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2259d-de1a-4fb6-8a70-9c87a4e4468a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a05bc43f32744c68619027e93a517c3, user_id=1c3869a453324112be04a5743a381f59, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:52.616608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2259d-de1a-4fb6-8a70-9c87a4e4468a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:52.657881 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 850/1691] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:55:52 2022] PUT /identity/v3/projects/2a05bc43f32744c68619027e93a517c3/users/1c3869a453324112be04a5743a381f59/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:55:53.745864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:53.747053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:53.747541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:53.836962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052f637-80dc-4083-8664-1f89bbb55ea8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:53.837941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052f637-80dc-4083-8664-1f89bbb55ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:53.838371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052f637-80dc-4083-8664-1f89bbb55ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:53.877198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] MFA Rules not processed for user `364cf4a7a3bf4e70973c78a5d690e998`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:53.878488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:53.879634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:53.884834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7052f637-80dc-4083-8664-1f89bbb55ea8 None None] MFA Rules not processed for user `03a3c989ecef42e084fb44d47c55b27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:53.886354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7052f637-80dc-4083-8664-1f89bbb55ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:54.050553 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 842/1692] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:54.063884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:54.065339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:54.066368 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 851/1693] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:54.068464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:54.068827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:54.069103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:54.096308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:54.100332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] MFA Rules not processed for user `03a3c989ecef42e084fb44d47c55b27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:54.103069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:54.115298 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 852/1694] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:54.133421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:54.134907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:54.241113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14f4b5-f4e2-42a7-8233-1ca3162292db None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:54.252332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3670e0-1988-4c38-be1e-cd65ffead40b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:54.252976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3670e0-1988-4c38-be1e-cd65ffead40b service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:54.253240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3670e0-1988-4c38-be1e-cd65ffead40b service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:54.253505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3670e0-1988-4c38-be1e-cd65ffead40b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:54.278244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3670e0-1988-4c38-be1e-cd65ffead40b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:54.279915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3670e0-1988-4c38-be1e-cd65ffead40b service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:54.316714 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 843/1695] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:55:54.327089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab36f-5fd8-49c4-bbde-e19ba8749b26 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:54.331581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab36f-5fd8-49c4-bbde-e19ba8749b26 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:54.331848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deab36f-5fd8-49c4-bbde-e19ba8749b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:54.339374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c14a6-ccde-425b-b6ee-7ad2dea642a0 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:54.366991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:54.383707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:54.383707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:54.383707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:54.427634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:54.436619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:54.486417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8deab36f-5fd8-49c4-bbde-e19ba8749b26 None None] MFA Rules not processed for user `081d365c7b15443aa76c7da2ca444a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:54.489383 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 853/1696] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:54 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:55:54.493312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8deab36f-5fd8-49c4-bbde-e19ba8749b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:54.710630 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 844/1697] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:54.792627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:54.808390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970dfde-8070-4506-91de-90d7e5c4c626 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:54.828759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b59d-de18-4edd-85a6-2b9550a57f06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:54.829512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b59d-de18-4edd-85a6-2b9550a57f06 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:54.829963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b59d-de18-4edd-85a6-2b9550a57f06 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:54.830340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b59d-de18-4edd-85a6-2b9550a57f06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:54.851513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b59d-de18-4edd-85a6-2b9550a57f06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:54.853293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54b59d-de18-4edd-85a6-2b9550a57f06 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:54.884996 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 854/1698] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:55.547393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a825-9a41-4b7e-855e-88b765fa0424 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:55.547979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a825-9a41-4b7e-855e-88b765fa0424 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:55.547979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360a825-9a41-4b7e-855e-88b765fa0424 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:55.550820 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 845/1699] 10.222.0.23 () {54 vars in 912 bytes} [Sun Sep 11 00:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:55.561118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:55.561509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:55.567244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:55.571986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1294-d939-402f-840a-4d3b3e889a02 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:55.572378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1294-d939-402f-840a-4d3b3e889a02 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:55.572711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1294-d939-402f-840a-4d3b3e889a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:55.619379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-052f1294-d939-402f-840a-4d3b3e889a02 None None] MFA Rules not processed for user `1c3869a453324112be04a5743a381f59`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:55.620981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-052f1294-d939-402f-840a-4d3b3e889a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:55.654893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:55.656172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:55.730513 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 855/1700] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:55.752586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:55.755939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:55.760760 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 846/1701] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:55.774130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:55.785032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361843e2-8241-4e37-b8d8-d957e61e8134 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:55.796074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27df39b-b5e7-47b6-b4cd-a33050b4e5a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:55.796917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27df39b-b5e7-47b6-b4cd-a33050b4e5a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:55.798237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27df39b-b5e7-47b6-b4cd-a33050b4e5a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:55.799884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27df39b-b5e7-47b6-b4cd-a33050b4e5a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:55.820434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27df39b-b5e7-47b6-b4cd-a33050b4e5a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:55.824884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27df39b-b5e7-47b6-b4cd-a33050b4e5a5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:55.851153 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 856/1702] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:56.092630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88affcb-d9a2-4ac9-9a72-ac3a2cecc836 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:56.093851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88affcb-d9a2-4ac9-9a72-ac3a2cecc836 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:56.094612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88affcb-d9a2-4ac9-9a72-ac3a2cecc836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:56.197426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c88affcb-d9a2-4ac9-9a72-ac3a2cecc836 None None] MFA Rules not processed for user `f06b14789a2740eca22001c0b5232fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:56.198865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c88affcb-d9a2-4ac9-9a72-ac3a2cecc836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:56.211041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:56.385888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:56.386496 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 847/1703] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:56.386646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:56.387217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:56.462799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] MFA Rules not processed for user `f06b14789a2740eca22001c0b5232fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:56.473182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:56.484905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458027-aa58-43a5-a1fc-cfad468264f9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:56.485085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458027-aa58-43a5-a1fc-cfad468264f9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:56.485394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99458027-aa58-43a5-a1fc-cfad468264f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:56.523676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-99458027-aa58-43a5-a1fc-cfad468264f9 None None] MFA Rules not processed for user `44c365d540894a908b34fe51d854e96e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:56.532904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-99458027-aa58-43a5-a1fc-cfad468264f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:56.557086 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 857/1704] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:56.564410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:56.609902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9555626-e2e3-4d0d-95e3-7314e04779dd None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:56.634606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:56.635461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:56.635847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:56.636315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:56.673486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:56.687089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:56.729808 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 858/1705] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:56.732274 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 848/1706] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:57.383526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:57.385026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:57.528706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d8ea0-44f7-4ab1-9929-179e9119f653 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:57.538812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:57.540190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:57.540496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:57.540769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:57.544629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:57.562236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:57.564055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:57.582117 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 859/1707] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:58.357393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a23cb-1e5d-4b1a-9a63-ea616c4f29b5 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:58.357909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a23cb-1e5d-4b1a-9a63-ea616c4f29b5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:58.357909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a23cb-1e5d-4b1a-9a63-ea616c4f29b5 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:58.359855 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 849/1708] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:58.426605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:58.473197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa73c4-d1c2-4c70-9fc8-f94ea4fb8210 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:58.473467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa73c4-d1c2-4c70-9fc8-f94ea4fb8210 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:58.474704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa73c4-d1c2-4c70-9fc8-f94ea4fb8210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:58.540519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2cfa73c4-d1c2-4c70-9fc8-f94ea4fb8210 None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:58.541657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2cfa73c4-d1c2-4c70-9fc8-f94ea4fb8210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:58.551001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9f80c-2d3e-4ae9-80eb-44f91d0fb0fd service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:58.569511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:58.573406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:58.574620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:58.575012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] PATH_INFO: `/v3/users/8fd2fe0504674e3cbbb73db14e894eb9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:58.578231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd2fe0504674e3cbbb73db14e894eb9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:58.578963 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 850/1709] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 11 00:55:58.581567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:58.686230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd2fe0504674e3cbbb73db14e894eb9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:58.714478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd2fe0504674e3cbbb73db14e894eb9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:58.725342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd2fe0504674e3cbbb73db14e894eb9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:58.727150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd2fe0504674e3cbbb73db14e894eb9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:58.730723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:58.776363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0070e040-4350-4aa3-9e58-1aac8accca2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd2fe0504674e3cbbb73db14e894eb9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:55:58.800423 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 860/1710] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:55:58 2022] DELETE /identity/v3/users/8fd2fe0504674e3cbbb73db14e894eb9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:55:58.805562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95ff14-bb2f-4c51-a5e7-ba4053bdf713 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:58.806238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95ff14-bb2f-4c51-a5e7-ba4053bdf713 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:58.807296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95ff14-bb2f-4c51-a5e7-ba4053bdf713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:58.859575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5a95ff14-bb2f-4c51-a5e7-ba4053bdf713 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:58.864410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5a95ff14-bb2f-4c51-a5e7-ba4053bdf713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:58.919068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55616fa4-5996-4997-87b2-94c9b5bdf16c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:58.919559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55616fa4-5996-4997-87b2-94c9b5bdf16c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:58.920314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55616fa4-5996-4997-87b2-94c9b5bdf16c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:58.922256 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 861/1711] 10.222.0.23 () {54 vars in 961 bytes} [Sun Sep 11 00:55:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:58.930983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:58.931415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:58.931824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:58.991686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:58.996129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:59.043652 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 851/1712] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:59.068077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:59.068607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:59.069034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:59.106374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] MFA Rules not processed for user `081d365c7b15443aa76c7da2ca444a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:59.110103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:59.190127 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 862/1713] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Sep 11 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:59.192394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:59.202083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:59.240474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:55:59.275861 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 852/1714] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:55:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:55:59.278515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:59.312480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320b1e8-5cd2-4613-901b-cc19927bf53a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:59.313502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e47de8b-ab3e-4e0a-912f-883dbf623810 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:59.318812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f361e4e-28d0-4407-a1bf-47ee962029eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:59.319827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361e4e-28d0-4407-a1bf-47ee962029eb service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:59.320349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361e4e-28d0-4407-a1bf-47ee962029eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:59.320874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f361e4e-28d0-4407-a1bf-47ee962029eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:59.333961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f361e4e-28d0-4407-a1bf-47ee962029eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:59.336130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f361e4e-28d0-4407-a1bf-47ee962029eb service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:59.337318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:59.337964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:59.338243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:59.338513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:59.344534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:59.359034 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 863/1715] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:59.360965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b550-e50b-4f39-9109-1ed69c797a0d None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:59.361452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b550-e50b-4f39-9109-1ed69c797a0d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:59.361953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b550-e50b-4f39-9109-1ed69c797a0d None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:59.363930 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 864/1716] 10.222.0.23 () {54 vars in 912 bytes} [Sun Sep 11 00:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:55:59.371438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeae74-54e1-4efd-8d22-bc52d89c5324 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:59.372052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeae74-54e1-4efd-8d22-bc52d89c5324 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:59.372506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eeae74-54e1-4efd-8d22-bc52d89c5324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:59.373876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:59.375511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:59.398156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-77eeae74-54e1-4efd-8d22-bc52d89c5324 None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:55:59.399906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-77eeae74-54e1-4efd-8d22-bc52d89c5324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:55:59.404427 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 853/1717] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:55:59.427234 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 865/1718] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:55:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:55:59.481221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:55:59.493746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615ba8-3c02-43ec-9a0f-241d55f73d7f service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:55:59.519724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:55:59.521520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:55:59.521703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:55:59.524450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:55:59.548170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:55:59.553758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:55:59.580096 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 854/1719] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:55:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:00.669810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efa5e-140e-463f-8434-ba055458e6e2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:00.671030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efa5e-140e-463f-8434-ba055458e6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:00.671481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efa5e-140e-463f-8434-ba055458e6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:00.708407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e13efa5e-140e-463f-8434-ba055458e6e2 None None] MFA Rules not processed for user `364cf4a7a3bf4e70973c78a5d690e998`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:00.718875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e13efa5e-140e-463f-8434-ba055458e6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:00.825217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:00.827089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:00.862132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27337d47-ac65-49c8-83ab-0fcdf6015e23 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:00.881719 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 866/1720] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:00.884128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:00.885072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:00.885566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:00.886035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] PATH_INFO: `/v3/projects/2d6af7325f8a486aa88c252d98e1a89e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:00.887483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6af7325f8a486aa88c252d98e1a89e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:00.889877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:00.948375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701562c4-8767-48fb-81d5-9f5de00784ea None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:00.949005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701562c4-8767-48fb-81d5-9f5de00784ea None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:00.949429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701562c4-8767-48fb-81d5-9f5de00784ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:01.006974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-701562c4-8767-48fb-81d5-9f5de00784ea None None] MFA Rules not processed for user `03a3c989ecef42e084fb44d47c55b27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:01.010612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-701562c4-8767-48fb-81d5-9f5de00784ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:01.036362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6af7325f8a486aa88c252d98e1a89e', 'request_id': 'req-b181b540-1d07-45a2-915c-f16f084f97bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.041176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:01.065182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6af7325f8a486aa88c252d98e1a89e', 'request_id': 'req-b181b540-1d07-45a2-915c-f16f084f97bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.075590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6af7325f8a486aa88c252d98e1a89e', 'request_id': 'req-b181b540-1d07-45a2-915c-f16f084f97bc'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.125014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] The token cache is being invalidate because project 2d6af7325f8a486aa88c252d98e1a89e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:01.126236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.128746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.145442 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 855/1721] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:00 2022] DELETE /identity/v3/projects/2d6af7325f8a486aa88c252d98e1a89e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:01.159891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:01.169634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:01.206054 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 867/1722] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:01.371126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181b540-1d07-45a2-915c-f16f084f97bc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:01.384087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:01.384827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:01.387691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:01.388291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] PATH_INFO: `/v3/users/358d645c2c6244d9bbd2143ba03e1941` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:01.391523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=358d645c2c6244d9bbd2143ba03e1941)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:01.396686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:01.499090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '358d645c2c6244d9bbd2143ba03e1941'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.503463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:01.523148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '358d645c2c6244d9bbd2143ba03e1941'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.562816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '358d645c2c6244d9bbd2143ba03e1941'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.565522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '358d645c2c6244d9bbd2143ba03e1941'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.582628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:01.582938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:01.583233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:01.594668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '358d645c2c6244d9bbd2143ba03e1941'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:01.624203 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 856/1723] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:01 2022] DELETE /identity/v3/users/358d645c2c6244d9bbd2143ba03e1941 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:01.713327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:01.716767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:01.929229 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 868/1724] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:56:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:02.014893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:02.016626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:02.323046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:02.324748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:02.404495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75b5cb-4c25-4a80-9ea9-85a0b520bf19 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:02.436692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:02.437528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:02.441790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:02.442928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] PATH_INFO: `/v3/users/5514476e82264e71ae2519fc3145ce1e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:02.447981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5514476e82264e71ae2519fc3145ce1e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:02.456487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:02.628945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5514476e82264e71ae2519fc3145ce1e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:02.662250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5514476e82264e71ae2519fc3145ce1e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:02.753352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99de2db9-f390-4d3a-8b95-3a803352b8bb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:02.754958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5514476e82264e71ae2519fc3145ce1e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:02.757617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5514476e82264e71ae2519fc3145ce1e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:02.764180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:02.765066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:02.765518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:02.767309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:02.770188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:02.780091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5514476e82264e71ae2519fc3145ce1e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:02.801964 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 857/1725] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:02 2022] DELETE /identity/v3/users/5514476e82264e71ae2519fc3145ce1e => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:02.803354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:02.806384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a6d1cc-6177-425a-a5f6-8ca8ff1e7d04 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:02.832053 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 869/1726] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:56:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:56:03.164726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:03.167287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:03.380403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:03.460470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad76c4-4a62-40a7-8c35-8e1c4afac513 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:03.481025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:03.484537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:03.485388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:03.486768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] PATH_INFO: `/v3/projects/e50a893eefa84b7198624e21c7a6e855` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:03.489885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50a893eefa84b7198624e21c7a6e855)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:03.496085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:03.670971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50a893eefa84b7198624e21c7a6e855', 'request_id': 'req-3585e1c9-6a0d-4351-a290-67c019f872b8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:03.696463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50a893eefa84b7198624e21c7a6e855', 'request_id': 'req-3585e1c9-6a0d-4351-a290-67c019f872b8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:03.705920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50a893eefa84b7198624e21c7a6e855', 'request_id': 'req-3585e1c9-6a0d-4351-a290-67c019f872b8'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:03.729049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] The token cache is being invalidate because project e50a893eefa84b7198624e21c7a6e855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:03.729426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:03.730104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3585e1c9-6a0d-4351-a290-67c019f872b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:03.736811 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 858/1727] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:03 2022] DELETE /identity/v3/projects/e50a893eefa84b7198624e21c7a6e855 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:03.738514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89261203-d60e-4978-8c15-436cca3f604c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:03.738789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89261203-d60e-4978-8c15-436cca3f604c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:03.739066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89261203-d60e-4978-8c15-436cca3f604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:03.767356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:03.788963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-89261203-d60e-4978-8c15-436cca3f604c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:03.790608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-89261203-d60e-4978-8c15-436cca3f604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:03.926110 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 870/1728] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:03.935771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:03.936155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:03.936359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:03.992512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:03.999019 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:04.085419 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 859/1729] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:04.086508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89261203-d60e-4978-8c15-436cca3f604c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:04.114166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89261203-d60e-4978-8c15-436cca3f604c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:04.131256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:04.142602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:04.142887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:04.143161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:04.143870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:04.151726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:04.156542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:04.214909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:04.216564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:04.221022 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 871/1730] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:04.226340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:04.259943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc8b9c-b07d-4d7d-a633-0af4e827b42b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:04.264334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354828d-7e92-4b3a-9610-5abab9db3b12 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:04.278692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:04.283584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:04.284160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:04.286924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:04.288427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:04.297745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:04.297971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:04.300482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:04.300760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:04.301024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] PATH_INFO: `/v3/projects/a16210ea6e5f48a3b6ede0b178e6622c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:04.305635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a16210ea6e5f48a3b6ede0b178e6622c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:04.320673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:04.415790 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 872/1731] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:04 2022] POST /identity/v3/projects => generated 365 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:04.432409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:04.698058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834904-5b69-4404-b4ea-edd227a70b22 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:04.718104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a16210ea6e5f48a3b6ede0b178e6622c', 'request_id': 'req-b36f9bcd-298c-48dc-b810-4bc97340469e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:04.746556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a16210ea6e5f48a3b6ede0b178e6622c', 'request_id': 'req-b36f9bcd-298c-48dc-b810-4bc97340469e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:04.764779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a16210ea6e5f48a3b6ede0b178e6622c', 'request_id': 'req-b36f9bcd-298c-48dc-b810-4bc97340469e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:04.831391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] The token cache is being invalidate because project a16210ea6e5f48a3b6ede0b178e6622c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:04.832149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:04.836721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:04.844383 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 860/1732] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:04 2022] DELETE /identity/v3/projects/a16210ea6e5f48a3b6ede0b178e6622c => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:04.848324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:04.853966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:04.945191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:04.946553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:04.947436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:04.948356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:04.950306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:04.954033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:05.017861 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 873/1733] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:04 2022] POST /identity/v3/users => generated 283 bytes in 584 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:05.020219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:05.025302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:05.059261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed8b8f-8103-48d8-9b71-28ac6e556978 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:05.073491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:05.075556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:05.075674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:05.075872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:05.078943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:05.084265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:05.092552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:05.158010 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 874/1734] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:05.160891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:05.166944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:05.220836 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1c671-365a-4679-9c51-a163c3c1d148 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:05.241465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f9bcd-298c-48dc-b810-4bc97340469e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:05.251430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:05.254144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:05.254144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:05.254144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:05.266966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:05.268709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:05.290653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:05.291665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:05.292014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:05.292350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] PATH_INFO: `/v3/users/1c3869a453324112be04a5743a381f59` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:05.297576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3869a453324112be04a5743a381f59)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:05.307004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:05.322352 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 875/1735] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:05.328630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:05.374934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c05c6-53b1-4a99-b8b2-8546e62d252f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:05.402531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:05.403409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:05.404144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:05.404144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] PATH_INFO: `/v3/projects/8a36b69b66aa4ab19ccfcb89b2dad8bf/users/3406322fcb5d4e0e9bd6306ee600fa12/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:05.444746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3869a453324112be04a5743a381f59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:05.456271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3869a453324112be04a5743a381f59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:05.488638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a36b69b66aa4ab19ccfcb89b2dad8bf, user_id=3406322fcb5d4e0e9bd6306ee600fa12, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:05.491026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:05.498343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3869a453324112be04a5743a381f59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:05.499605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3869a453324112be04a5743a381f59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:05.524411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c67b00c2-a13e-4223-bc72-dc9d213ef391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3869a453324112be04a5743a381f59'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:05.537462 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 861/1736] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:04 2022] DELETE /identity/v3/users/1c3869a453324112be04a5743a381f59 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:05.541563 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 876/1737] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:05 2022] PUT /identity/v3/projects/8a36b69b66aa4ab19ccfcb89b2dad8bf/users/3406322fcb5d4e0e9bd6306ee600fa12/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:05.553827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa8a30-298a-4e17-a820-f302d58c80c4 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:05.554476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa8a30-298a-4e17-a820-f302d58c80c4 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:05.555004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa8a30-298a-4e17-a820-f302d58c80c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:05.609848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f1aa8a30-298a-4e17-a820-f302d58c80c4 None None] MFA Rules not processed for user `3406322fcb5d4e0e9bd6306ee600fa12`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:05.611445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f1aa8a30-298a-4e17-a820-f302d58c80c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:05.785792 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 862/1738] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:05.790437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:05.793220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:05.981470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66342c61-985a-41fa-b2c6-5a2dc2a0a88b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:06.154678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:06.155504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:06.155929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:06.156300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:06.157043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:06.159215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:06.223034 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 877/1739] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:05 2022] POST /identity/v3/projects => generated 367 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:06.226778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:06.227066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:06.227386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:06.233059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:06.311313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:06.312528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:06.519122 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:06.571703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056871d-6e9c-4c61-b21b-22e5dbdbf81c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:06.584214 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 863/1740] 10.222.0.46 () {58 vars in 1015 bytes} [Sun Sep 11 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:06.594334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:06.595033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:06.597992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:06.598440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:06.599966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:06.616283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:06.689535 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 878/1741] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:06 2022] POST /identity/v3/users => generated 290 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:06.693641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:06.717854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd296fb-9c79-4efa-b8d4-b052a4216e4d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:06.730171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:06.730899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:06.731260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:06.731538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:06.732252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:06.733844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:06.747574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:06.773731 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 864/1742] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:06.775635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:06.789483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52274d-8259-4f1c-bfa7-30ba86923b27 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:06.814448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:06.815497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:06.816004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:06.816474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] PATH_INFO: `/v3/projects/91d0e1f2aed74b53ac44c9dd55f85dd0/users/867904414f184c3ea0180f26cf15929f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:06.873176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d0e1f2aed74b53ac44c9dd55f85dd0, user_id=867904414f184c3ea0180f26cf15929f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:06.884815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:06.927004 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 879/1743] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:06 2022] PUT /identity/v3/projects/91d0e1f2aed74b53ac44c9dd55f85dd0/users/867904414f184c3ea0180f26cf15929f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:06.931366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:06.946346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a8a40-04ec-45c0-a9a9-e23c36f72d7d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:07.124291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:07.126798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:07.148793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:07.149812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:07.152225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:07.152857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:07.153792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:07.158430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:07.201971 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 865/1744] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:07.206457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:07.253017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cd7fc-9476-43ed-bb7d-94846f86e17f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:07.277716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe583fe9-f609-4636-aec2-0dee31f997d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:07.279706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe583fe9-f609-4636-aec2-0dee31f997d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:07.279985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe583fe9-f609-4636-aec2-0dee31f997d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:07.280553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe583fe9-f609-4636-aec2-0dee31f997d7 admin admin] PATH_INFO: `/v3/projects/91d0e1f2aed74b53ac44c9dd55f85dd0/users/867904414f184c3ea0180f26cf15929f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:07.292827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe583fe9-f609-4636-aec2-0dee31f997d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d0e1f2aed74b53ac44c9dd55f85dd0, user_id=867904414f184c3ea0180f26cf15929f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:07.299535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe583fe9-f609-4636-aec2-0dee31f997d7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:07.372418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0180-8a0f-4d09-9fd2-4e4fe9975009 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:07.386034 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 866/1745] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:07 2022] PUT /identity/v3/projects/91d0e1f2aed74b53ac44c9dd55f85dd0/users/867904414f184c3ea0180f26cf15929f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:07.392887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:07.393174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:07.393450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:07.573472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] MFA Rules not processed for user `867904414f184c3ea0180f26cf15929f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:07.573867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:07.574810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:07.575277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:07.575820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:07.576044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:07.599277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:07.601183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:07.625192 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 880/1746] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:07.823371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:07.829456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:07.876559 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 867/1747] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:07.897652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:07.900652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:08.069164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f2f05-55ff-49d6-ac36-46f66b28b446 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:08.106001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:08.106826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.120271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.120953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] PATH_INFO: `/v3/projects/2a05bc43f32744c68619027e93a517c3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.134525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a05bc43f32744c68619027e93a517c3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:08.136914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c6b76-2c11-4f49-b2b0-e90c4a9a5d7c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:08.140676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:08.155434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a9c06-d020-4cf3-b9c3-f767501b7c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:08.156238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9c06-d020-4cf3-b9c3-f767501b7c1e service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.158428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9c06-d020-4cf3-b9c3-f767501b7c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.158806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a9c06-d020-4cf3-b9c3-f767501b7c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.211572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9c06-d020-4cf3-b9c3-f767501b7c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:08.211572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a9c06-d020-4cf3-b9c3-f767501b7c1e service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:08.224783 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:08.257658 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 868/1748] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:08.328564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a05bc43f32744c68619027e93a517c3', 'request_id': 'req-d9363c35-6151-4900-a478-c67469fa7b4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:08.361363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a05bc43f32744c68619027e93a517c3', 'request_id': 'req-d9363c35-6151-4900-a478-c67469fa7b4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:08.379850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a05bc43f32744c68619027e93a517c3', 'request_id': 'req-d9363c35-6151-4900-a478-c67469fa7b4e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:08.423228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] The token cache is being invalidate because project 2a05bc43f32744c68619027e93a517c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:08.423787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:08.426927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:08.430487 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 881/1749] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:07 2022] DELETE /identity/v3/projects/2a05bc43f32744c68619027e93a517c3 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:08.436139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd2f59-88ef-427e-af03-ee1f16a3a7a2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.437178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd2f59-88ef-427e-af03-ee1f16a3a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.437626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd2f59-88ef-427e-af03-ee1f16a3a7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.494449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b7dd2f59-88ef-427e-af03-ee1f16a3a7a2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:08.495935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b7dd2f59-88ef-427e-af03-ee1f16a3a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:08.514270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:08.521356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:08.522999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:08.674961 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 869/1750] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:08.685598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.686645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.687291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.715010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9363c35-6151-4900-a478-c67469fa7b4e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:08.733774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:08.734683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.735116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.735423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.745446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:08.753647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:08.754684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:08.800951 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 870/1751] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:08.802083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:08.820418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeefe8-4ff3-4e33-aacd-246c8b2f81fa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:08.843854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:08.843854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.843854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.843854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.844388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:08.857931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:08.861077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:08.944414 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 871/1752] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:08.946651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:08.974967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561af7ae-002f-4038-b7aa-5e17ca24980c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:08.979142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:08.984846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:08.993837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:08.993837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:08.994137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:08.994372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:08.995008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:09.001650 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:09.022256 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 882/1753] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:09.147560 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 872/1754] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:08 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:09.149119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:09.194612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80665095-1ff2-42a9-9702-dda31c41e80e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:09.414184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:09.415700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:09.416344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:09.418033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:09.420131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:09.425020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:09.489019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:09.545062 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 883/1755] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:09 2022] POST /identity/v3/users => generated 293 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:09.546391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:09.572541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36616d52-1eb7-4b89-8ac0-8039ac511856 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:09.598890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:09.599913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:09.600397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:09.600876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:09.601727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:09.604277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:09.651238 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 873/1756] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:09.652688 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:09.686182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef21af-cd5c-471a-be8d-59ce50b1e5b6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:09.700049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50742490-2ede-46e6-b0dc-6e3d96820e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:09.701642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50742490-2ede-46e6-b0dc-6e3d96820e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:09.702679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50742490-2ede-46e6-b0dc-6e3d96820e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:09.704120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50742490-2ede-46e6-b0dc-6e3d96820e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:09.705037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50742490-2ede-46e6-b0dc-6e3d96820e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:09.711052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50742490-2ede-46e6-b0dc-6e3d96820e63 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:09.766306 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 884/1757] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:09.767700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:09.788161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36557fa7-ba91-455e-aaf3-b4fdb939740b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:09.808388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:09.809145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:09.809503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:09.809779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] PATH_INFO: `/v3/projects/209b7b282ac948a8998a95712cc3dac4/users/b0bb2ce3595b46a389f993d61732dc9d/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:09.869152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209b7b282ac948a8998a95712cc3dac4, user_id=b0bb2ce3595b46a389f993d61732dc9d, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:09.888072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:09.942560 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 874/1758] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:09 2022] PUT /identity/v3/projects/209b7b282ac948a8998a95712cc3dac4/users/b0bb2ce3595b46a389f993d61732dc9d/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:09.944950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:09.945233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:09.945512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:10.040245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] MFA Rules not processed for user `b0bb2ce3595b46a389f993d61732dc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:10.041737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:10.277167 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 885/1759] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:10.281739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:10.526852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8aa8bd-0aa6-4321-8907-fb45e38af5a3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:10.568845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:10.569555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:10.570352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:10.570352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:10.570698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:10.595593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:10.779577 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 875/1760] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:10 2022] POST /identity/v3/projects => generated 387 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:10.780859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:10.811754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8b813-6bb3-4494-b4bd-7653d69701da None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:11.078912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334665a-32a9-44f4-9934-57369cafdf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:11.081687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334665a-32a9-44f4-9934-57369cafdf0f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.082992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334665a-32a9-44f4-9934-57369cafdf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.083300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334665a-32a9-44f4-9934-57369cafdf0f admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.083972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334665a-32a9-44f4-9934-57369cafdf0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:11.093395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334665a-32a9-44f4-9934-57369cafdf0f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:11.156453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:11.170188 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 886/1761] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:10 2022] POST /identity/v3/users => generated 300 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:11.180951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f4421-eba8-4f28-8eb7-38f3036a582c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:11.196085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253da68a-abe4-468e-a95d-581b0a3f8fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:11.200931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d39c-0fb5-4e54-9764-55993ae59630 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.202970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d39c-0fb5-4e54-9764-55993ae59630 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.203445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d39c-0fb5-4e54-9764-55993ae59630 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.205657 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 887/1762] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:56:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:56:11.206391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253da68a-abe4-468e-a95d-581b0a3f8fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.211480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253da68a-abe4-468e-a95d-581b0a3f8fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.212095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253da68a-abe4-468e-a95d-581b0a3f8fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.212957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253da68a-abe4-468e-a95d-581b0a3f8fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:11.218551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253da68a-abe4-468e-a95d-581b0a3f8fbd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:11.254130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d39c-0fb5-4e54-9764-55993ae59630 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:11.258505 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 876/1763] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:11.297900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d39c-0fb5-4e54-9764-55993ae59630 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:11.318682 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:11.321124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.321405 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.321677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] PATH_INFO: `/v3/projects/fc3a7086d96e404b8f88b4722e076836/users/4b0236f510534d32943fb97ffa8af90d/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.334524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.334801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.335403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.374259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:11.404277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:11.412670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:11.415600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3a7086d96e404b8f88b4722e076836, user_id=4b0236f510534d32943fb97ffa8af90d, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:11.432071 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:11.476988 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 888/1764] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:11 2022] PUT /identity/v3/projects/fc3a7086d96e404b8f88b4722e076836/users/4b0236f510534d32943fb97ffa8af90d/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:11.479916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:11.646510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:11.724000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34febde8-1c5c-4a29-afdd-c1ff984cd771 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:11.749424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05ca14-fbc0-4a6d-929f-87e67ef19c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:11.750268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05ca14-fbc0-4a6d-929f-87e67ef19c66 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.750630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05ca14-fbc0-4a6d-929f-87e67ef19c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.751598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05ca14-fbc0-4a6d-929f-87e67ef19c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.753408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05ca14-fbc0-4a6d-929f-87e67ef19c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:11.760865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05ca14-fbc0-4a6d-929f-87e67ef19c66 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:11.799776 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 877/1765] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:11.808115 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 889/1766] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:11.813238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:11.841092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384606a-adfc-41a6-af51-79abdb4bf415 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:11.848231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:11.852416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.853085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.853662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] PATH_INFO: `/v3/projects/fc3a7086d96e404b8f88b4722e076836/users/4b0236f510534d32943fb97ffa8af90d/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:11.858175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3a7086d96e404b8f88b4722e076836, user_id=4b0236f510534d32943fb97ffa8af90d, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:11.865687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:11.891156 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 878/1767] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:11 2022] PUT /identity/v3/projects/fc3a7086d96e404b8f88b4722e076836/users/4b0236f510534d32943fb97ffa8af90d/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:11.902650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:11.903587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:11.903900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:12.034843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] MFA Rules not processed for user `4b0236f510534d32943fb97ffa8af90d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:12.036168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:12.273431 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 890/1768] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:12.289025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:12.291494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:12.499561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2311ad-6095-4a27-ae67-8cb9e0f4a461 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:12.517144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d236dc-ee21-4e55-be4a-503ea35781ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:12.518046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d236dc-ee21-4e55-be4a-503ea35781ad service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:12.518324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d236dc-ee21-4e55-be4a-503ea35781ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:12.518725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d236dc-ee21-4e55-be4a-503ea35781ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:12.550089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d236dc-ee21-4e55-be4a-503ea35781ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:12.551584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d236dc-ee21-4e55-be4a-503ea35781ad service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:12.590388 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 879/1769] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:12.650878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:12.992180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2c796-1139-4ad9-b4f6-7c0c75ce34ec None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:13.030428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:13.031075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:13.033634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:13.034079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] PATH_INFO: `/v3/users/b0bb2ce3595b46a389f993d61732dc9d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:13.035728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0bb2ce3595b46a389f993d61732dc9d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:13.041812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:13.269623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0bb2ce3595b46a389f993d61732dc9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:13.304208 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0bb2ce3595b46a389f993d61732dc9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:13.327342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:13.346898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0bb2ce3595b46a389f993d61732dc9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:13.354332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0bb2ce3595b46a389f993d61732dc9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:13.405512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0bb2ce3595b46a389f993d61732dc9d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:13.418111 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 891/1770] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:12 2022] DELETE /identity/v3/users/b0bb2ce3595b46a389f993d61732dc9d => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:13.428561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:13.429219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:13.429471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:13.430492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:13.432387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:13.496621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:13.498193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:13.659836 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100d033-e8b1-4cc3-b6a2-b7b2e9a61ac5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:13.683593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:13.685807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:13.685807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:13.685807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:13.692286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:13.696900 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 880/1771] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:13.711343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:13.714031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:13.807925 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 892/1772] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:13.964524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:13.966230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:14.052569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:14.168778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064be6e-16fa-48a1-ae3d-60f4559a7efa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:14.182345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dda141-e316-4985-a6d5-f2d6d616d3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:14.185585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda141-e316-4985-a6d5-f2d6d616d3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:14.189458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda141-e316-4985-a6d5-f2d6d616d3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:14.189846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda141-e316-4985-a6d5-f2d6d616d3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:14.226139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dda141-e316-4985-a6d5-f2d6d616d3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:14.230780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dda141-e316-4985-a6d5-f2d6d616d3c6 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:14.263365 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 881/1773] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:16.847351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:16.850423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:16.897612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64754dd1-8461-4d5a-a038-fd15d803ba7a service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:16.904809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:16.905469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:16.905745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:16.906029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] PATH_INFO: `/v3/users/3406322fcb5d4e0e9bd6306ee600fa12` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:16.908958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3406322fcb5d4e0e9bd6306ee600fa12)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:16.912418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:17.030035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3406322fcb5d4e0e9bd6306ee600fa12'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:17.048656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3406322fcb5d4e0e9bd6306ee600fa12'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:17.073230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3406322fcb5d4e0e9bd6306ee600fa12'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:17.074813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3406322fcb5d4e0e9bd6306ee600fa12'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:17.113050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3406322fcb5d4e0e9bd6306ee600fa12'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:17.130876 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 893/1774] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:16 2022] DELETE /identity/v3/users/3406322fcb5d4e0e9bd6306ee600fa12 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:17.137255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:17.138358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:17.138358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:17.181550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:17.184897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:17.347789 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 882/1775] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:17.550328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:17.552099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:17.687633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:17.732390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2264f-eb55-401e-81bc-276bee74419b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:17.736542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117906bd-9c4e-4ccd-8897-efb9df0058a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:17.737169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117906bd-9c4e-4ccd-8897-efb9df0058a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:17.737443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117906bd-9c4e-4ccd-8897-efb9df0058a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:17.738437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117906bd-9c4e-4ccd-8897-efb9df0058a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:17.749926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117906bd-9c4e-4ccd-8897-efb9df0058a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:17.752886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117906bd-9c4e-4ccd-8897-efb9df0058a5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:17.765969 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 894/1776] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:18.680410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:18.682420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:18.701982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d734d4e-febf-4f74-9e75-65ff142f2058 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:18.707946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:18.709466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:18.709898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:18.710306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] PATH_INFO: `/v3/users/eb838183737c40728ed9e105d17461ea` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:18.711914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb838183737c40728ed9e105d17461ea)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:18.714911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:18.796470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb838183737c40728ed9e105d17461ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:18.807240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb838183737c40728ed9e105d17461ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:18.824187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb838183737c40728ed9e105d17461ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:18.824940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb838183737c40728ed9e105d17461ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:18.839314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb838183737c40728ed9e105d17461ea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:18.853291 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 883/1777] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:18 2022] DELETE /identity/v3/users/eb838183737c40728ed9e105d17461ea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:18.857760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:18.858281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:18.858586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:18.881154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:18.882386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:18.962035 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 895/1778] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:18.972318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:18.974053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:19.027066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:19.040494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff285a6b-8d35-49cb-9414-ac00df55ea96 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:19.044916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:19.045551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:19.045877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:19.046148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:19.056933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:19.058414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:19.069078 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 884/1779] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:19.164085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:19.167328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:19.180795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8b2a7-8fa8-44ce-89fe-035aa2d82e9f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:19.186448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:19.187259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:19.187597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:19.187926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] PATH_INFO: `/v3/projects/209b7b282ac948a8998a95712cc3dac4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:19.189183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209b7b282ac948a8998a95712cc3dac4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:19.191148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:19.255244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209b7b282ac948a8998a95712cc3dac4', 'request_id': 'req-ff303a04-8112-457e-b2c7-ac0d102b7283'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.264944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209b7b282ac948a8998a95712cc3dac4', 'request_id': 'req-ff303a04-8112-457e-b2c7-ac0d102b7283'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.274360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209b7b282ac948a8998a95712cc3dac4', 'request_id': 'req-ff303a04-8112-457e-b2c7-ac0d102b7283'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.295807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] The token cache is being invalidate because project 209b7b282ac948a8998a95712cc3dac4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:19.296119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.297247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.304682 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 896/1780] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:19 2022] DELETE /identity/v3/projects/209b7b282ac948a8998a95712cc3dac4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:19.305687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:19.337630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47dd02-cb6c-47a4-a164-77bf7acb7550 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:19.426571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:19.427422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:19.427701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:19.427973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] PATH_INFO: `/v3/users/4b0236f510534d32943fb97ffa8af90d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:19.429194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b0236f510534d32943fb97ffa8af90d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:19.431145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:19.502933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b0236f510534d32943fb97ffa8af90d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.516844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b0236f510534d32943fb97ffa8af90d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.543618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b0236f510534d32943fb97ffa8af90d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.545195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b0236f510534d32943fb97ffa8af90d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.563245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b0236f510534d32943fb97ffa8af90d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.566377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:19.568329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:19.575655 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 885/1781] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:19 2022] DELETE /identity/v3/users/4b0236f510534d32943fb97ffa8af90d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:19.592476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:19.594356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:19.602374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff303a04-8112-457e-b2c7-ac0d102b7283 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:19.678210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:19.678866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:19.679216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:19.680518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] PATH_INFO: `/v3/projects/8a36b69b66aa4ab19ccfcb89b2dad8bf` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:19.682702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a36b69b66aa4ab19ccfcb89b2dad8bf)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:19.685947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:19.693215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:19.779997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e31ec3-4c0e-4b9d-b4ab-a56cabfc6f44 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:19.795924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a36b69b66aa4ab19ccfcb89b2dad8bf', 'request_id': 'req-44c42925-f3af-45dc-b25c-5818dbc42a34'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.806920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:19.815003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a36b69b66aa4ab19ccfcb89b2dad8bf', 'request_id': 'req-44c42925-f3af-45dc-b25c-5818dbc42a34'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.831850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a36b69b66aa4ab19ccfcb89b2dad8bf', 'request_id': 'req-44c42925-f3af-45dc-b25c-5818dbc42a34'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.856893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] The token cache is being invalidate because project 8a36b69b66aa4ab19ccfcb89b2dad8bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:19.857929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.859653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.862173 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 897/1782] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:19 2022] DELETE /identity/v3/projects/8a36b69b66aa4ab19ccfcb89b2dad8bf => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:19.866983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:19.869068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:19.891261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:19.892590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:19.893072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:19.893542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] PATH_INFO: `/v3/projects/63444830fe5d444ab62aeb91347eb229` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:19.895904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=63444830fe5d444ab62aeb91347eb229)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:19.900059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:19.975003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c42925-f3af-45dc-b25c-5818dbc42a34 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:19.981248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63444830fe5d444ab62aeb91347eb229', 'request_id': 'req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:19.996146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63444830fe5d444ab62aeb91347eb229', 'request_id': 'req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.008143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63444830fe5d444ab62aeb91347eb229', 'request_id': 'req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.030568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] The token cache is being invalidate because project 63444830fe5d444ab62aeb91347eb229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:20.031004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.032043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.038495 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 886/1783] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:19 2022] DELETE /identity/v3/projects/63444830fe5d444ab62aeb91347eb229 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:20.040391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:20.043850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:20.094572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:20.095367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:20.095643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:20.095913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] PATH_INFO: `/v3/users/867904414f184c3ea0180f26cf15929f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:20.099556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=867904414f184c3ea0180f26cf15929f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:20.103657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:20.152996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a31e4-47f8-429a-9520-2f49c9c41d6c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:20.164480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:20.165030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:20.165305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:20.165665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] PATH_INFO: `/v3/users/95f52de4d444423eadab4b1086c8817f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:20.166960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95f52de4d444423eadab4b1086c8817f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:20.174694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:20.195679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:20.206187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867904414f184c3ea0180f26cf15929f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.220651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867904414f184c3ea0180f26cf15929f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.233531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867904414f184c3ea0180f26cf15929f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.235109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867904414f184c3ea0180f26cf15929f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.253966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c246ae91-57ba-415c-9687-0a8968c0e743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867904414f184c3ea0180f26cf15929f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.264495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:20.273721 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 898/1784] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:19 2022] DELETE /identity/v3/users/867904414f184c3ea0180f26cf15929f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:20.286785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95f52de4d444423eadab4b1086c8817f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.310680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95f52de4d444423eadab4b1086c8817f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.330627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95f52de4d444423eadab4b1086c8817f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.332276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95f52de4d444423eadab4b1086c8817f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.342641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9090de69-cae2-4867-a775-fef2a485624e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95f52de4d444423eadab4b1086c8817f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:20.352338 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 887/1785] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:20 2022] DELETE /identity/v3/users/95f52de4d444423eadab4b1086c8817f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:20.513778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3908-adde-4ed3-ab8c-bfcaf5320338 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:20.514487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3908-adde-4ed3-ab8c-bfcaf5320338 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:20.514822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a3908-adde-4ed3-ab8c-bfcaf5320338 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:20.516469 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 899/1786] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:56:20.562711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:20.563101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:20.563605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:20.589817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:20.591519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:20.705859 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 888/1787] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:20.852985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a3908-adde-4ed3-ab8c-bfcaf5320338 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:20.855110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5b6a3908-adde-4ed3-ab8c-bfcaf5320338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:20.913171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:20.916913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.019156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05714e61-0f9f-4873-9a3c-fffd81e3ee9c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.033930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a3908-adde-4ed3-ab8c-bfcaf5320338 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.034917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.036174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.036701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.038482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] PATH_INFO: `/v3/projects/913d8a5f819a4fbcae46614659f44363` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.041252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.042092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.042520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.042941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] PATH_INFO: `/v3/projects/91d0e1f2aed74b53ac44c9dd55f85dd0` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.043570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=913d8a5f819a4fbcae46614659f44363)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.044977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d0e1f2aed74b53ac44c9dd55f85dd0)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.049567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.055033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.129612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d0e1f2aed74b53ac44c9dd55f85dd0', 'request_id': 'req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.139241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913d8a5f819a4fbcae46614659f44363', 'request_id': 'req-5242810b-ada8-4476-96a6-edb0a9352854'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.144841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d0e1f2aed74b53ac44c9dd55f85dd0', 'request_id': 'req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.162555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913d8a5f819a4fbcae46614659f44363', 'request_id': 'req-5242810b-ada8-4476-96a6-edb0a9352854'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.164952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:21.166511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d0e1f2aed74b53ac44c9dd55f85dd0', 'request_id': 'req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.178457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913d8a5f819a4fbcae46614659f44363', 'request_id': 'req-5242810b-ada8-4476-96a6-edb0a9352854'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.193594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] The token cache is being invalidate because project 91d0e1f2aed74b53ac44c9dd55f85dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:21.195016 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.196005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-bd100a10-cd7b-4fce-af03-ff3021ae0fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.198103 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 900/1788] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:20 2022] DELETE /identity/v3/projects/91d0e1f2aed74b53ac44c9dd55f85dd0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:21.200240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] The token cache is being invalidate because project 913d8a5f819a4fbcae46614659f44363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:21.200539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.201333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5242810b-ada8-4476-96a6-edb0a9352854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.202873 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 889/1789] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:20 2022] DELETE /identity/v3/projects/913d8a5f819a4fbcae46614659f44363 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:21.203352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd173b9e-b049-4ef3-9a2d-b53937e2b2c6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.204557 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd173b9e-b049-4ef3-9a2d-b53937e2b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.204845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd173b9e-b049-4ef3-9a2d-b53937e2b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.211302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66257c05-8257-4808-ba9f-c13f105d388b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.211582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66257c05-8257-4808-ba9f-c13f105d388b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.211856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66257c05-8257-4808-ba9f-c13f105d388b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.229830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cd173b9e-b049-4ef3-9a2d-b53937e2b2c6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:21.232052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cd173b9e-b049-4ef3-9a2d-b53937e2b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.235188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-66257c05-8257-4808-ba9f-c13f105d388b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:21.236634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-66257c05-8257-4808-ba9f-c13f105d388b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.291284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:21.323450 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 901/1790] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:21.325983 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 890/1791] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:21.329240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.329530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.330173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.336016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.336306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.336579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.363798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:21.367201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.384159 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 902/1792] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:21.388968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.392258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:21.393789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.402727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c667d7-e295-41b1-8e89-3bcc8f181571 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.404502 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 891/1793] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:21.408941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.411868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.412729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.413251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.413701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.414586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:21.415055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.417160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.439150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cdb05-0aa0-49cb-8872-6a0d99f309e5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.443032 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 903/1794] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:56:21.447372 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.448152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.450132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.450762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.451604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.452796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:21.453538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.457181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.465730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fd753-7d0a-43ba-a3bc-8d187571fb3e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.475376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.476384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.476836 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 892/1795] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:56:21.481083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.483630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.484136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.485064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.489081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.492525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db0f87-743e-4a39-ab7e-2819335c6230 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.499248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.500590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.500590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.500838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.501652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.503709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.536713 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 904/1796] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:21.538108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.541199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.555513 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 893/1797] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:21.556900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.660867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a6b2c-df2f-409d-b851-614b0a3c5251 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.666407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3887866-4f7b-4138-bbdf-5e73ad298cc9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.679561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.679561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.679561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.679872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.681065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.683130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.707903 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 894/1798] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/users => generated 291 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:21.711952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.723088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4bc32-57bd-4144-904c-ac2ef0f1c862 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.728015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.728689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.728956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.729219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.730056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.732492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.760999 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 895/1799] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:21 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:21.762886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.768278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.768985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.769261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.770652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.773086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.777565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d14b6b-8ee6-44e7-8efd-3342e5c37df0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.786506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.787421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.787706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.788031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.788849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.791100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.796885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.799478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.813679 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 905/1800] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:21.818865 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 896/1801] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:21.824370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.826073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.827663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:21.832963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:21.838181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932c2b9-d50e-4b12-992d-4c694f58be72 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.846083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.846758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.847271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.847720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.848799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.850238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e2797-8a43-46a7-8949-cb7c89896156 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.850860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.856842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.857514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.857741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.858570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] PATH_INFO: `/v3/projects/fc3a7086d96e404b8f88b4722e076836` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.859861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3a7086d96e404b8f88b4722e076836)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.862629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.867716 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 897/1802] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:21.868832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.888359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e80c3d-0039-4c8b-abd9-fb7beaa70033 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.893819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:21.894548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:21.894916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:21.895380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:21.895514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:21.897010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:21.899298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:21.922210 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 898/1803] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:21.922210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.931452 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37159647-bef2-4df8-94f4-fe8c3f238672 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:21.936330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3a7086d96e404b8f88b4722e076836', 'request_id': 'req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.947786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3a7086d96e404b8f88b4722e076836', 'request_id': 'req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.956657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3a7086d96e404b8f88b4722e076836', 'request_id': 'req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.969621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] The token cache is being invalidate because project fc3a7086d96e404b8f88b4722e076836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:21.970185 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.971073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:21.977395 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 906/1804] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:21 2022] DELETE /identity/v3/projects/fc3a7086d96e404b8f88b4722e076836 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:21.977395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:21.977395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:22.038484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46a98a-983a-425c-b415-0701b83c277b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.039107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46a98a-983a-425c-b415-0701b83c277b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.039460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46a98a-983a-425c-b415-0701b83c277b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.040035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46a98a-983a-425c-b415-0701b83c277b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.040697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46a98a-983a-425c-b415-0701b83c277b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.042932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46a98a-983a-425c-b415-0701b83c277b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.057174 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 899/1805] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:22.058469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.058753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.059117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.079576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:22.080808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:22.083984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e563c0-2fd4-465a-bb2f-2a52172e5fc9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.089873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6079b-5eb0-4d0a-851b-893c2779485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.090052 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 900/1806] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:22.091020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6079b-5eb0-4d0a-851b-893c2779485b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.091020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6079b-5eb0-4d0a-851b-893c2779485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.091260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6079b-5eb0-4d0a-851b-893c2779485b admin admin] PATH_INFO: `/v3/projects/5615dcfe4e814ad582aedde318b41d18/users/cbef3251892a42bf88d0a16e8d0a2f6a/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.098154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.099942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:22.112723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66493d74-1b89-49e8-841c-a266956d35cd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.118683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ba09f2-1744-4094-935c-299c06408452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.118855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6079b-5eb0-4d0a-851b-893c2779485b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5615dcfe4e814ad582aedde318b41d18, user_id=cbef3251892a42bf88d0a16e8d0a2f6a, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.119539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba09f2-1744-4094-935c-299c06408452 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.119886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba09f2-1744-4094-935c-299c06408452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.120237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba09f2-1744-4094-935c-299c06408452 admin admin] PATH_INFO: `/v3/projects/5fcd185e7a4045e3912adcbdf7646f5e/users/dcb78d686f2645f589baa7444d88bb30/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.122002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6079b-5eb0-4d0a-851b-893c2779485b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.141258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba09f2-1744-4094-935c-299c06408452 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fcd185e7a4045e3912adcbdf7646f5e, user_id=dcb78d686f2645f589baa7444d88bb30, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.144245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ba09f2-1744-4094-935c-299c06408452 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.157412 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 907/1807] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:21 2022] PUT /identity/v3/projects/5615dcfe4e814ad582aedde318b41d18/users/cbef3251892a42bf88d0a16e8d0a2f6a/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:22.157412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200fdd7-4116-4a13-a5b0-1bfb8007998d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.157412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200fdd7-4116-4a13-a5b0-1bfb8007998d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.157412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6200fdd7-4116-4a13-a5b0-1bfb8007998d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.173878 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 901/1808] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:22 2022] PUT /identity/v3/projects/5fcd185e7a4045e3912adcbdf7646f5e/users/dcb78d686f2645f589baa7444d88bb30/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:22.175413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.175794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.176159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.190911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6200fdd7-4116-4a13-a5b0-1bfb8007998d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:22.192729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6200fdd7-4116-4a13-a5b0-1bfb8007998d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:22.216074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] MFA Rules not processed for user `cbef3251892a42bf88d0a16e8d0a2f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:22.217427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:22.310813 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 908/1809] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:22.310813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.310813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.310813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.315656 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 902/1810] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:22.327761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.356764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4d202-810a-47d2-a876-2a8f2a52737b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.360021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] MFA Rules not processed for user `dcb78d686f2645f589baa7444d88bb30`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:22.361432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:22.362747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.363858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.364277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.364837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.366217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:22.366677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.369704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.389076 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 903/1811] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:22.390387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.397174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:22.414205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1c644d-2b7a-4df9-8475-a011dc801563 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.424149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.425016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.425457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.425876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.427228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.430582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.466407 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 909/1812] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:22.467583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.470422 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 904/1813] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:22.475654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.479668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b3870-6460-4f99-a86f-07bda82f4afe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.481036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:22.556544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.557282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.557625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.558300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.559439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.562288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.604715 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 910/1814] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:22.605735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fdcd25-39ab-404b-9775-3b1b518e3f41 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.606178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.674816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.675608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.676037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.676670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.677742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.679830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.707292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72060c47-caa0-4bb1-a6e3-3b7b520fe49d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.715888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.717087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.717566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.718022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.719335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.720157 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 905/1815] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:22.721209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.723669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.757788 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 911/1816] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/users => generated 298 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:22.757788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.768847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18411b89-08fa-4683-ba6d-a5165c57ef13 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.802083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968613b0-1de1-4909-8b38-e1033e510533 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.825489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.825489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.825489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.825489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.826305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.826305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.841600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:22.856900 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 906/1817] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:22.858085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.870125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.871165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.871554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35627c-fcda-4e26-8aa5-47e2a045c6c1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.871975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.872408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.873180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.875555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.877779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.878688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.879053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.879471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.880346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.882222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.899791 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 907/1818] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:22.901122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.907827 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 912/1819] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:22 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:22.908776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.918123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfc734-7a4f-41dc-bb7e-faf8a47a37c8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.919102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf0f26-6548-43b7-ac47-60e8a68bf011 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.924311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.924895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.925100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.925435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.925745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.925838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.926654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.926654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] PATH_INFO: `/v3/projects/849de36b91e54a78b2bc5a695af16013/users/56db802bec734015adc79ca82c375fca/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.926857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.928375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.940454 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 908/1820] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:22.941393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.942909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=849de36b91e54a78b2bc5a695af16013, user_id=56db802bec734015adc79ca82c375fca, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.946109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.946918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:22.952918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ba4cf-2a0d-4c41-a594-fcdfedfd29dc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:22.962140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:22.962376 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 913/1821] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:22 2022] PUT /identity/v3/projects/849de36b91e54a78b2bc5a695af16013/users/56db802bec734015adc79ca82c375fca/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:22.963040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:22.963364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:22.963674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:22.963772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:22.964436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:22.967192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:22.987935 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 909/1822] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:22.988636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.002261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a537c-1c5e-447b-9254-b2fe5742203b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.053019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34b4bd-80b4-41fd-b28c-d80ec45c19aa admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.058210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.058897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.059268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.059605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.060470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.062131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.076482 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 914/1823] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:23.081997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.085463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.086425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.086854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.087541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.088774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.091152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.096966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69309685-23ab-4539-84fc-80656766e6ba admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.102335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.102944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.103256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.103529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.104916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.106608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.107312 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 910/1824] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:23.109223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.127248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ede88c-d3bb-4916-bbb9-385c34cbf155 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.128668 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 915/1825] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:23.129882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.132655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee876d9-8de4-43fc-af86-7936cff10086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.133401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee876d9-8de4-43fc-af86-7936cff10086 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.133738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee876d9-8de4-43fc-af86-7936cff10086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.134602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee876d9-8de4-43fc-af86-7936cff10086 admin admin] PATH_INFO: `/v3/projects/87af636cbb7c467095335c7840e34e8e/users/9e492403adfc48fc97cd3f4877630ec7/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.147605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745dc88-f971-4456-aa1f-40d37024ce74 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.155480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee876d9-8de4-43fc-af86-7936cff10086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87af636cbb7c467095335c7840e34e8e, user_id=9e492403adfc48fc97cd3f4877630ec7, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.156926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.157702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.158159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.158441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] PATH_INFO: `/v3/projects/849de36b91e54a78b2bc5a695af16013/users/56db802bec734015adc79ca82c375fca/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.159488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee876d9-8de4-43fc-af86-7936cff10086 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.161078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] RBAC: Authorizing `identity:create_grant(project_id=849de36b91e54a78b2bc5a695af16013, user_id=56db802bec734015adc79ca82c375fca, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.164108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.185763 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 916/1826] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:23 2022] PUT /identity/v3/projects/849de36b91e54a78b2bc5a695af16013/users/56db802bec734015adc79ca82c375fca/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:23.188119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.189971 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 911/1827] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:23 2022] PUT /identity/v3/projects/87af636cbb7c467095335c7840e34e8e/users/9e492403adfc48fc97cd3f4877630ec7/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:23.197795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76fbb1-26aa-4896-bdc3-f3ce166a26cc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.198040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76fbb1-26aa-4896-bdc3-f3ce166a26cc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.198290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76fbb1-26aa-4896-bdc3-f3ce166a26cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.231450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0f76fbb1-26aa-4896-bdc3-f3ce166a26cc None None] MFA Rules not processed for user `56db802bec734015adc79ca82c375fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:23.233009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f76fbb1-26aa-4896-bdc3-f3ce166a26cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:23.262378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efec76-4ff6-445e-aef3-67f248e5264b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.288098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:23.334414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.335345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.335716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.336125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] PATH_INFO: `/v3/projects/372f508af07a4704b4619c68ead6cccc/users/ce07423d6dd04b6391b81ac748f45429/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.362471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=372f508af07a4704b4619c68ead6cccc, user_id=ce07423d6dd04b6391b81ac748f45429, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.363876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.371868 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 912/1828] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:23.373723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bd62-5af4-4be6-ba0a-bfc43f64a05f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.373723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bd62-5af4-4be6-ba0a-bfc43f64a05f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.373936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434bd62-5af4-4be6-ba0a-bfc43f64a05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.400048 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 917/1829] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:23 2022] PUT /identity/v3/projects/372f508af07a4704b4619c68ead6cccc/users/ce07423d6dd04b6391b81ac748f45429/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:23.401766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.402664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:23.431148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c434bd62-5af4-4be6-ba0a-bfc43f64a05f None None] MFA Rules not processed for user `9e492403adfc48fc97cd3f4877630ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:23.432397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c434bd62-5af4-4be6-ba0a-bfc43f64a05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:23.508600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:23.512703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a263e-4460-4b5a-b844-33ab9a2cec9e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.521537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.523771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.524179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.524549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.566152 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 913/1830] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:23.567655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.568037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.568311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.599592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.600399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] MFA Rules not processed for user `ce07423d6dd04b6391b81ac748f45429`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:23.601051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.601630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:23.612425 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 918/1831] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:23.613478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.702521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:23.711778 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 914/1832] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:23.716172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.717122 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e829b75-e3b2-458c-b6ac-1921196f4f61 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.723852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.724599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.725021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.725412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.726206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.728376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.734736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1b217-9573-4251-88c6-2d180afd439b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.742106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.743121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.743601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.744038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.744878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.747494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.764440 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 919/1833] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:23.765504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.791701 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 915/1834] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:23.793428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.839674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640542d8-d6ee-4c0b-9cce-6ebb3f800f6d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.880864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588a9ea-680e-45dd-bece-0af3e7cd8ef0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.886517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.887503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.887854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.888218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.888964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.891826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.900622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.901564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.902019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.902436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.933588 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 916/1835] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:23.935074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:23.962754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900e540-e1e9-4f9a-86a1-2b661108f99b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:23.973430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:23.975834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:23.976223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:23.976589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:23.977788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.980586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.980694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:23.981744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:23.992435 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 920/1836] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:23.993298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.007344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83990dad-baa4-4f27-b03f-e1a24dcab057 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.013914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.014616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.015244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.015244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.015909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.017812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.037563 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 917/1837] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:23 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:24.039472 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 921/1838] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:24.046249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.054549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.061669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cedfaf-3282-473c-b232-4373af8666be admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.067223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.068265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d61be-9cbc-45a4-b953-562b168c5075 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.070455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.070455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.070455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.070693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.072937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.076719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.077749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.078244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.078792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] PATH_INFO: `/v3/projects/7f49d1243ba349a996b8c20053142615/users/fcc0971c78124bf78d8796bfcfad595e/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.088619 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 918/1839] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:24.092862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.100355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f49d1243ba349a996b8c20053142615, user_id=fcc0971c78124bf78d8796bfcfad595e, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.103848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.109755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f6eba-eabb-4346-9a99-da8a694e947d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.117814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.118917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.118917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.119106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] PATH_INFO: `/v3/projects/877fd038a34a468e90e6dcd5b28205c2/users/5fb32ad613d84efcbd63b2545ab7a40e/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.122330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:24.156468 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 922/1840] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:24 2022] PUT /identity/v3/projects/7f49d1243ba349a996b8c20053142615/users/fcc0971c78124bf78d8796bfcfad595e/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:24.160322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.166989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=877fd038a34a468e90e6dcd5b28205c2, user_id=5fb32ad613d84efcbd63b2545ab7a40e, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.171191 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.211583 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 919/1841] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:24 2022] PUT /identity/v3/projects/877fd038a34a468e90e6dcd5b28205c2/users/5fb32ad613d84efcbd63b2545ab7a40e/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:24.216118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.240083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:24.305220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9be8e-3841-4399-8eb3-b3ff3eebd3e4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.312315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.312946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.313221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.313607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.314265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.316108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.327101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05865-47da-4342-b2ec-fe70839c3c47 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.335799 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 920/1842] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:24.337317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.340059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.340739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.341047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.341287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.342049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.348971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.361522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37ed76-d037-4a05-a3cf-dedc317a8013 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.368244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba46f9-793c-4d9f-8901-52c1d95f8440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.368928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba46f9-793c-4d9f-8901-52c1d95f8440 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.369206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba46f9-793c-4d9f-8901-52c1d95f8440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.369476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba46f9-793c-4d9f-8901-52c1d95f8440 admin admin] PATH_INFO: `/v3/projects/877fd038a34a468e90e6dcd5b28205c2/users/5fb32ad613d84efcbd63b2545ab7a40e/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.372205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba46f9-793c-4d9f-8901-52c1d95f8440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=877fd038a34a468e90e6dcd5b28205c2, user_id=5fb32ad613d84efcbd63b2545ab7a40e, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.377984 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 923/1843] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:24.378929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.380099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba46f9-793c-4d9f-8901-52c1d95f8440 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.396213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4640c4-cd43-43e2-be8f-5f805a70c26a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.411731 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 921/1844] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:24 2022] PUT /identity/v3/projects/877fd038a34a468e90e6dcd5b28205c2/users/5fb32ad613d84efcbd63b2545ab7a40e/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:24.422357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f914bc-c919-475a-914a-db5cca85a658 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.423042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f914bc-c919-475a-914a-db5cca85a658 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.423681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f914bc-c919-475a-914a-db5cca85a658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.477912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d9f914bc-c919-475a-914a-db5cca85a658 None None] MFA Rules not processed for user `5fb32ad613d84efcbd63b2545ab7a40e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:24.479415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d9f914bc-c919-475a-914a-db5cca85a658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:24.485997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.486777 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.487052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.487348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] PATH_INFO: `/v3/projects/7f49d1243ba349a996b8c20053142615/users/fcc0971c78124bf78d8796bfcfad595e/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.489412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f49d1243ba349a996b8c20053142615, user_id=fcc0971c78124bf78d8796bfcfad595e, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.492427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.506249 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 924/1845] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:24 2022] PUT /identity/v3/projects/7f49d1243ba349a996b8c20053142615/users/fcc0971c78124bf78d8796bfcfad595e/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:24.507906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.592857 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 922/1846] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:24.594453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.595467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.596297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.638801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] MFA Rules not processed for user `fcc0971c78124bf78d8796bfcfad595e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:24.640533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:24.652542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12bc79-9694-4a46-bc54-6841eee8acd1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.659354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.660448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.661066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.661680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] PATH_INFO: `/v3/users/cbef3251892a42bf88d0a16e8d0a2f6a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.663234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbef3251892a42bf88d0a16e8d0a2f6a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.665525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.707298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:24.730067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbef3251892a42bf88d0a16e8d0a2f6a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:24.733577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:24.742685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbef3251892a42bf88d0a16e8d0a2f6a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:24.754285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbef3251892a42bf88d0a16e8d0a2f6a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:24.755671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbef3251892a42bf88d0a16e8d0a2f6a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:24.755814 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 923/1847] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:24.757211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.778834 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbef3251892a42bf88d0a16e8d0a2f6a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:24.785390 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 925/1848] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:24 2022] DELETE /identity/v3/users/cbef3251892a42bf88d0a16e8d0a2f6a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:24.786171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:24.787971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:24.860557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129fa49-d9fb-4ef3-a7df-4c529b541d8f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.877907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.879794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.880633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.881771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.882545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ed349-6b33-4121-aef4-b02e0d40a1f0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:24.885053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:24.887795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:24.888879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.889365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.889776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.892035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:24.959926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.961364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18137710-fc61-42c1-9762-c1b31018b387 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.972947 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 924/1849] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:24.974294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:24.974581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:24.974858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:24.991222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:24.993116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:24.997361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:24.998590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.004596 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 926/1850] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:25.117333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:25.121941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.126808 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 925/1851] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:25.142012 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:25.144392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.153497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddd410-a8d1-4e7e-963a-9af2925d7e31 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:25.167035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:25.167955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:25.168338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:25.168731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] PATH_INFO: `/v3/users/9e492403adfc48fc97cd3f4877630ec7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:25.171077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e492403adfc48fc97cd3f4877630ec7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:25.175452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:25.201578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:25.242903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac6f19-860c-48af-80c9-d479440d07d2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:25.246767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e492403adfc48fc97cd3f4877630ec7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.247248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e80807-6e85-4937-bb0a-87307ee9f9b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:25.247999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e80807-6e85-4937-bb0a-87307ee9f9b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:25.248358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e80807-6e85-4937-bb0a-87307ee9f9b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:25.248912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e80807-6e85-4937-bb0a-87307ee9f9b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:25.261582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e492403adfc48fc97cd3f4877630ec7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.263393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e80807-6e85-4937-bb0a-87307ee9f9b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:25.267755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e80807-6e85-4937-bb0a-87307ee9f9b1 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:25.273264 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e492403adfc48fc97cd3f4877630ec7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.274039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e492403adfc48fc97cd3f4877630ec7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.289191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:25.290861 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 926/1852] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:25.301197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e492403adfc48fc97cd3f4877630ec7'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.314881 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 927/1853] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:25 2022] DELETE /identity/v3/users/9e492403adfc48fc97cd3f4877630ec7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:25.316853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:25.317196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:25.317533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:25.354064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:25.355741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.463703 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 927/1854] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:25.478129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:25.490128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.592778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d903c-8cdf-4dd0-9f4a-2580905f0b6b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:25.597446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:25.598196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:25.599520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:25.599520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:25.609479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:25.611140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:25.634561 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 928/1855] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:25.693595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:25.695143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.713674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bdc2e-84f6-43a1-95df-859e175afedf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:25.722851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:25.724810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:25.725315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:25.725948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] PATH_INFO: `/v3/projects/5615dcfe4e814ad582aedde318b41d18` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:25.727784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5615dcfe4e814ad582aedde318b41d18)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:25.731304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:25.788818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:25.790257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.803194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5615dcfe4e814ad582aedde318b41d18', 'request_id': 'req-6da7e884-c67c-49e4-b323-e7d128a67b14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.814798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5615dcfe4e814ad582aedde318b41d18', 'request_id': 'req-6da7e884-c67c-49e4-b323-e7d128a67b14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.828399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5615dcfe4e814ad582aedde318b41d18', 'request_id': 'req-6da7e884-c67c-49e4-b323-e7d128a67b14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.849459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] The token cache is being invalidate because project 5615dcfe4e814ad582aedde318b41d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:25.849719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.850535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:25.858882 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 928/1856] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:25 2022] DELETE /identity/v3/projects/5615dcfe4e814ad582aedde318b41d18 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:25.859880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:25.862606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:25.926045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e58c2d-e815-46b8-8f8d-0ed35d106eeb service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:25.961669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:25.984830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7e884-c67c-49e4-b323-e7d128a67b14 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:26.006240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:26.007016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.007360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.007637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] PATH_INFO: `/v3/users/56db802bec734015adc79ca82c375fca` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.009298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=56db802bec734015adc79ca82c375fca)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:26.011734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:26.036243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:26.036881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.037160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.037448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.039440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:26.099504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56db802bec734015adc79ca82c375fca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.111509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56db802bec734015adc79ca82c375fca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.131214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56db802bec734015adc79ca82c375fca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.132379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56db802bec734015adc79ca82c375fca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.152557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c13fd9b2-f1c6-488c-9d6b-2cd9a09cd07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56db802bec734015adc79ca82c375fca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.170655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:26.172186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc010d8b-f623-4b57-b467-04271e4ce33e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:26.178332 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 929/1857] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:25 2022] DELETE /identity/v3/users/56db802bec734015adc79ca82c375fca => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:26.181100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93959f-1ed3-4d0a-89ad-6dac65c4dc88 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.181750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93959f-1ed3-4d0a-89ad-6dac65c4dc88 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.182165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93959f-1ed3-4d0a-89ad-6dac65c4dc88 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.183989 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 930/1858] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:56:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:56:26.184938 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 929/1859] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:26.199153 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93959f-1ed3-4d0a-89ad-6dac65c4dc88 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:26.205003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff93959f-1ed3-4d0a-89ad-6dac65c4dc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:26.239613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.239613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.240072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.254089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:26.296332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:26.297578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:26.410509 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 930/1860] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:26.416874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93959f-1ed3-4d0a-89ad-6dac65c4dc88 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:26.426900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:26.427997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.428348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.428656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] PATH_INFO: `/v3/projects/87af636cbb7c467095335c7840e34e8e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.430140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87af636cbb7c467095335c7840e34e8e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:26.432727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:26.506877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87af636cbb7c467095335c7840e34e8e', 'request_id': 'req-1fc25269-c0d3-40ff-bc29-04923a72dc22'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.519026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87af636cbb7c467095335c7840e34e8e', 'request_id': 'req-1fc25269-c0d3-40ff-bc29-04923a72dc22'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.529228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87af636cbb7c467095335c7840e34e8e', 'request_id': 'req-1fc25269-c0d3-40ff-bc29-04923a72dc22'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.566139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] The token cache is being invalidate because project 87af636cbb7c467095335c7840e34e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:26.566435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.567641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.576946 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 931/1861] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:26 2022] DELETE /identity/v3/projects/87af636cbb7c467095335c7840e34e8e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:26.579421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:26.593821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd632-c35d-4255-b7d2-fae460c10ed1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:26.694323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:26.696152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:26.696828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:26.697949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.698455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.698841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] PATH_INFO: `/v3/users/fcc0971c78124bf78d8796bfcfad595e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.700256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc0971c78124bf78d8796bfcfad595e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:26.702375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:26.702647 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:26.808753 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc0971c78124bf78d8796bfcfad595e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.826341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc0971c78124bf78d8796bfcfad595e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.840324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc0971c78124bf78d8796bfcfad595e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.841204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc0971c78124bf78d8796bfcfad595e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.859352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc0971c78124bf78d8796bfcfad595e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:26.876479 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 931/1862] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:26 2022] DELETE /identity/v3/users/fcc0971c78124bf78d8796bfcfad595e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:26.877500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:26.878192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc25269-c0d3-40ff-bc29-04923a72dc22 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:26.883762 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:26.902574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:26.975785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:26.976269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:26.976359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:26.976588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:26.982579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:26.993037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865043-76c5-4ea3-bb75-b6855b7ae88b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:26.996735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:26.998817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c769ec3d-6799-4648-a843-03f1bf70444f service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.000918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.002045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.003698 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.005028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] PATH_INFO: `/v3/projects/849de36b91e54a78b2bc5a695af16013` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.008089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=849de36b91e54a78b2bc5a695af16013)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.010538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.023697 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 932/1863] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:27.078665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '849de36b91e54a78b2bc5a695af16013', 'request_id': 'req-c2f8374a-c16a-4c82-9087-e165303afc20'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.090109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '849de36b91e54a78b2bc5a695af16013', 'request_id': 'req-c2f8374a-c16a-4c82-9087-e165303afc20'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.114461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '849de36b91e54a78b2bc5a695af16013', 'request_id': 'req-c2f8374a-c16a-4c82-9087-e165303afc20'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.137063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] The token cache is being invalidate because project 849de36b91e54a78b2bc5a695af16013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:27.137366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.138191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c2f8374a-c16a-4c82-9087-e165303afc20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.145548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.145733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.145984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.149430 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 932/1864] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:26 2022] DELETE /identity/v3/projects/849de36b91e54a78b2bc5a695af16013 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:27.176064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:27.178233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:27.299773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.300354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.300455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.306341 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 933/1865] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:27.328949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:27.330445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:27.348067 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 933/1866] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:27.350907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.368095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539fba9-a8e0-436c-a7b7-d1c604a65707 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.371337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.375229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:27.376164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.377299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.377887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.378300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.379154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:27.379629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.381455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.404883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74536d7-0e8f-4401-b4a5-cd5c0488cfc3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.407225 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 934/1867] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:27.410964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.414526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.415138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.415511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.415857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] PATH_INFO: `/v3/projects/7f49d1243ba349a996b8c20053142615` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.417501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f49d1243ba349a996b8c20053142615)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.420178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.434161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea5a4c-e493-4327-b4c7-954177b293aa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.450705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.451666 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.453323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.454135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.455696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.458435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.486008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:27.498623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f49d1243ba349a996b8c20053142615', 'request_id': 'req-6568d288-3c79-4470-bd22-5d5f933a11f6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.509154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f49d1243ba349a996b8c20053142615', 'request_id': 'req-6568d288-3c79-4470-bd22-5d5f933a11f6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.517567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f49d1243ba349a996b8c20053142615', 'request_id': 'req-6568d288-3c79-4470-bd22-5d5f933a11f6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.518950 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 935/1868] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:27.522934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.533555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] The token cache is being invalidate because project 7f49d1243ba349a996b8c20053142615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:27.534003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.534790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6568d288-3c79-4470-bd22-5d5f933a11f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:27.536849 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 934/1869] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:27 2022] DELETE /identity/v3/projects/7f49d1243ba349a996b8c20053142615 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:27.542217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b6e71-2201-4401-bb09-96fb1eb4b185 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.542511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b6e71-2201-4401-bb09-96fb1eb4b185 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.542975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b6e71-2201-4401-bb09-96fb1eb4b185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.569843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7b2b6e71-2201-4401-bb09-96fb1eb4b185 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:27.571091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7b2b6e71-2201-4401-bb09-96fb1eb4b185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:27.591581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:27.623308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833541b-e490-41f4-82c5-6dc5392e18cb admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.629255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.629893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.630167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.630434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.631759 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.634782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.672349 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 936/1870] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:27.673073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.673276 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 935/1871] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:27.675134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:27.675594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.676036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.676563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.693837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e767b436-8f73-4cfa-b857-def5a240f2dd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.719403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.720690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.721116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.721634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.723904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.726717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.746242 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 937/1872] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:27.750072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.752390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:27.754597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:27.768290 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 936/1873] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:27.769567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.777277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c04a9c-c750-4344-8538-8694b4097321 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.790642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0cfa5-d006-4364-a7b9-1c7506b2bab6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.791484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.792207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.792617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.793294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.794749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.797273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.809069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.809705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.809974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.810240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.810902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:27.811224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.812858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.836723 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 938/1874] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:27.837818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.853141 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 937/1875] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:27.857353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:27.859017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f3d93b-c412-4411-9948-0f7e97924aa5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.873590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16050e-5d09-4cdb-96dd-affb2c9ab693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.874425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16050e-5d09-4cdb-96dd-affb2c9ab693 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.874806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16050e-5d09-4cdb-96dd-affb2c9ab693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.875218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16050e-5d09-4cdb-96dd-affb2c9ab693 admin admin] PATH_INFO: `/v3/projects/e063ad95dc8644e8a5ea9c38c75eb990/users/57c2cdd9626146a1881a3d1b04509cde/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.876759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d8117-05fe-4397-b4fb-f7330d5c60cc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:27.891221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:27.894168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.894614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.895078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.895946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.899583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.916667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16050e-5d09-4cdb-96dd-affb2c9ab693 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e063ad95dc8644e8a5ea9c38c75eb990, user_id=57c2cdd9626146a1881a3d1b04509cde, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:27.920048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16050e-5d09-4cdb-96dd-affb2c9ab693 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:27.960655 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 939/1876] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:27 2022] PUT /identity/v3/projects/e063ad95dc8644e8a5ea9c38c75eb990/users/57c2cdd9626146a1881a3d1b04509cde/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:27.974395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:27.974744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:27.975644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:27.984299 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 938/1877] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:27.987527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.047421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] MFA Rules not processed for user `57c2cdd9626146a1881a3d1b04509cde`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:28.048578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:28.171962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da444c-9d1b-4253-996c-2aba8bad6162 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.178707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.179971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.180106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.180564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.181227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.184744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.233027 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 940/1878] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:28.242694 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 939/1879] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:27 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:28.249245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.249594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.254794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:28.271820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a58a9-5055-4119-aca6-b9507b13b7dd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.286117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.287325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.287664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.287935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.288665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.290350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.309212 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 940/1880] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:28.314231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.329841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:28.336286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86791146-810b-4659-8ccf-cc066b1d48f0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.343632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.344524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.345050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.345473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.346503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.349202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.366870 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 941/1881] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:28.372103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.388727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cad53a-70c7-4955-a942-02144df46bc9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.395360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b473dc9-966e-4b15-bd6d-176073bfb67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.396761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b473dc9-966e-4b15-bd6d-176073bfb67d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.397129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b473dc9-966e-4b15-bd6d-176073bfb67d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.398132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b473dc9-966e-4b15-bd6d-176073bfb67d admin admin] PATH_INFO: `/v3/projects/4898b96fc51d4acca67f8c26b1e46518/users/2e0d8bebf5a44fe2b54380a46405b0dd/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.413148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0dc5e4-e10b-4989-acc4-d80c5b8dd0a7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.423824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b473dc9-966e-4b15-bd6d-176073bfb67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4898b96fc51d4acca67f8c26b1e46518, user_id=2e0d8bebf5a44fe2b54380a46405b0dd, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.424822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.425621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.426002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.426707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.429725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b473dc9-966e-4b15-bd6d-176073bfb67d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.443382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.444956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.456671 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 942/1882] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:28 2022] PUT /identity/v3/projects/4898b96fc51d4acca67f8c26b1e46518/users/2e0d8bebf5a44fe2b54380a46405b0dd/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:28.471300 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 941/1883] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:28.471607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.471607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.471607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.516198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] MFA Rules not processed for user `2e0d8bebf5a44fe2b54380a46405b0dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:28.517589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:28.528074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:28.608346 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 943/1884] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:28.608960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.620828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f0e07-8b6e-449d-a446-b751c7fd6bf3 service glance] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.685942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.686562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.686835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.687098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.687764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.689710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.719989 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 942/1885] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:28 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:28.723950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.734877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d821ca0-0571-4dda-9906-cfd75c42ce26 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.802691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.804052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.804052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.804253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.804883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.806825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.833251 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 944/1886] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:28 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:28.837437 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.856735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58627c7-2d56-43c7-a61e-0490751f37a9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.863766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.864388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.864664 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.864924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.865602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.867236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.880079 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 943/1887] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:28.885001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.896067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e246545-8744-4cb1-a4a4-a46f8c6bea88 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:28.908649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:28.909615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:28.909765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:28.910107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] PATH_INFO: `/v3/projects/221e0e666a2a496789bf3a61d56e0de1/users/f9e9ceff67894e2f90eb78850f640872/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:28.932646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221e0e666a2a496789bf3a61d56e0de1, user_id=f9e9ceff67894e2f90eb78850f640872, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:28.935570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:28.951552 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 945/1888] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:28 2022] PUT /identity/v3/projects/221e0e666a2a496789bf3a61d56e0de1/users/f9e9ceff67894e2f90eb78850f640872/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:28.956540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:28.967732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df0df-c663-4a0f-902e-9013310b32e3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:29.040586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a36f5-fc8e-4308-8652-47206960fe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:29.041617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a36f5-fc8e-4308-8652-47206960fe09 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:29.041987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a36f5-fc8e-4308-8652-47206960fe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:29.042353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a36f5-fc8e-4308-8652-47206960fe09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:29.043102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a36f5-fc8e-4308-8652-47206960fe09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:29.048299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a36f5-fc8e-4308-8652-47206960fe09 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:29.063702 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 944/1889] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:29.067267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:29.079247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60817202-fab0-4c92-b550-0b67a5f90fe1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:29.085537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:29.086221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:29.086622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:29.086905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] PATH_INFO: `/v3/projects/221e0e666a2a496789bf3a61d56e0de1/users/f9e9ceff67894e2f90eb78850f640872/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:29.089004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] RBAC: Authorizing `identity:create_grant(project_id=221e0e666a2a496789bf3a61d56e0de1, user_id=f9e9ceff67894e2f90eb78850f640872, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:29.091859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:29.114626 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 946/1890] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:29 2022] PUT /identity/v3/projects/221e0e666a2a496789bf3a61d56e0de1/users/f9e9ceff67894e2f90eb78850f640872/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:29.125327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:29.125672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:29.126073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:29.156590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] MFA Rules not processed for user `f9e9ceff67894e2f90eb78850f640872`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:29.158035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:29.165658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:29.252162 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 945/1891] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:29.259953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:29.261868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:29.380473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3d687-2ee1-4a06-9926-15b23e4e637a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:29.393215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520011c-f54e-45ec-a849-3e0e45f161c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:29.393916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520011c-f54e-45ec-a849-3e0e45f161c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:29.394265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520011c-f54e-45ec-a849-3e0e45f161c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:29.394601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520011c-f54e-45ec-a849-3e0e45f161c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:29.424168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520011c-f54e-45ec-a849-3e0e45f161c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:29.425662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520011c-f54e-45ec-a849-3e0e45f161c9 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:29.445566 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 947/1892] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:30.257290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:30.327903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb510f52-a421-4597-adf3-8b989f04e1fb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:30.332789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:30.333437 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:30.333714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:30.333981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] PATH_INFO: `/v3/users/2e0d8bebf5a44fe2b54380a46405b0dd` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:30.335120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e0d8bebf5a44fe2b54380a46405b0dd)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:30.337160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:30.395531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e0d8bebf5a44fe2b54380a46405b0dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:30.405936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e0d8bebf5a44fe2b54380a46405b0dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:30.427780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e0d8bebf5a44fe2b54380a46405b0dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:30.428735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e0d8bebf5a44fe2b54380a46405b0dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:30.435912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e0d8bebf5a44fe2b54380a46405b0dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:30.443848 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 946/1893] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:30 2022] DELETE /identity/v3/users/2e0d8bebf5a44fe2b54380a46405b0dd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:30.448413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280517fe-6979-4d85-88b1-85543825a89a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:30.448981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280517fe-6979-4d85-88b1-85543825a89a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:30.448981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280517fe-6979-4d85-88b1-85543825a89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:30.462357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:30.470146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-280517fe-6979-4d85-88b1-85543825a89a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:30.472395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-280517fe-6979-4d85-88b1-85543825a89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:30.536526 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 948/1894] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:30.546884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:30.548451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:30.578521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:30.623688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f498f1-36b6-45cb-a5ad-8ca4f005d9f8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:30.628435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:30.629318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:30.629585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:30.629983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:30.641229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:30.642820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:30.654869 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 947/1895] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:30.971820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280517fe-6979-4d85-88b1-85543825a89a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:30.973259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-280517fe-6979-4d85-88b1-85543825a89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:30.986845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280517fe-6979-4d85-88b1-85543825a89a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:30.992487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:30.993195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:30.993623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:30.993942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] PATH_INFO: `/v3/projects/4898b96fc51d4acca67f8c26b1e46518` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:30.995360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4898b96fc51d4acca67f8c26b1e46518)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:31.000761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:31.026319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:31.027844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:31.052087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4898b96fc51d4acca67f8c26b1e46518', 'request_id': 'req-ed2319a4-9f7c-4d63-b1e2-4545799819de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.060472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4898b96fc51d4acca67f8c26b1e46518', 'request_id': 'req-ed2319a4-9f7c-4d63-b1e2-4545799819de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.066962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4898b96fc51d4acca67f8c26b1e46518', 'request_id': 'req-ed2319a4-9f7c-4d63-b1e2-4545799819de'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.077483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] The token cache is being invalidate because project 4898b96fc51d4acca67f8c26b1e46518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:31.077866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.078614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.080393 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 949/1896] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:30 2022] DELETE /identity/v3/projects/4898b96fc51d4acca67f8c26b1e46518 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:31.084020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:31.085341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:31.097167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd7d10-8340-42da-aeac-c53ee22763dc service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:31.159655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:31.160384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:31.160781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:31.161049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:31.162760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:31.169160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2319a4-9f7c-4d63-b1e2-4545799819de admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:31.174502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:31.175144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:31.175445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:31.176154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] PATH_INFO: `/v3/users/f9e9ceff67894e2f90eb78850f640872` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:31.176953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e9ceff67894e2f90eb78850f640872)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:31.178877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:31.228781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:31.230596 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:31.240648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:31.257047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e9ceff67894e2f90eb78850f640872'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.259039 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 948/1897] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:56:31.266893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e9ceff67894e2f90eb78850f640872'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.274719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e9ceff67894e2f90eb78850f640872'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.276015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e9ceff67894e2f90eb78850f640872'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.289904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-361f0965-73bf-4511-9cad-5e3292481eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e9ceff67894e2f90eb78850f640872'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:31.296964 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 950/1898] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:31 2022] DELETE /identity/v3/users/f9e9ceff67894e2f90eb78850f640872 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:31.825405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:31.837935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d16d27-86cb-4ee9-9a62-b2dd1d8bea9e service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:31.893553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:31.942216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:31.944713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:31.945096 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:31.946375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] PATH_INFO: `/v3/projects/221e0e666a2a496789bf3a61d56e0de1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:31.951447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=221e0e666a2a496789bf3a61d56e0de1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:31.953509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.020884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221e0e666a2a496789bf3a61d56e0de1', 'request_id': 'req-705e7bac-e617-4181-b178-a381d8db2ebf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:32.044695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221e0e666a2a496789bf3a61d56e0de1', 'request_id': 'req-705e7bac-e617-4181-b178-a381d8db2ebf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:32.051247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221e0e666a2a496789bf3a61d56e0de1', 'request_id': 'req-705e7bac-e617-4181-b178-a381d8db2ebf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:32.064453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] The token cache is being invalidate because project 221e0e666a2a496789bf3a61d56e0de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:32.064453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:32.065395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-705e7bac-e617-4181-b178-a381d8db2ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:32.067532 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 949/1899] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:31 2022] DELETE /identity/v3/projects/221e0e666a2a496789bf3a61d56e0de1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:32.072938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.073519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.073519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.101504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:32.105431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:32.180770 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 951/1900] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:32.190007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.190291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.190586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.217634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:32.218902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:32.228476 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 950/1901] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:32.232938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.246971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7c534-1bfe-4c7a-ae59-15e754ae2982 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.268434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.269366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.269956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.270507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.271389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:32.271740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.273510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.288402 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 952/1902] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:32.293290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.303788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686de26-4c00-4495-b327-81c18590c2a3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.312725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.313354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.313624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.313889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.314517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.316480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.353794 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 951/1903] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:32.355719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.367046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b49f9c-96a0-4f51-9208-a27a7ba3b011 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.430361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.431056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.431356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.431623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.432262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.434175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.461949 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 953/1904] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:32.462660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.478471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156b221-5524-4e43-ba27-d0577d3cc1b6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.486681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.487428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.487753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.488023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.488726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.491057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.506984 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 952/1905] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:32.508349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.520026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11605c2-7d1c-4ba9-94cd-266215e3eca2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.525620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481d54-ee48-47f2-8bf6-0f37bca1e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.526449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481d54-ee48-47f2-8bf6-0f37bca1e333 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.526850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481d54-ee48-47f2-8bf6-0f37bca1e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.527385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481d54-ee48-47f2-8bf6-0f37bca1e333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.528151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f481d54-ee48-47f2-8bf6-0f37bca1e333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.530059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f481d54-ee48-47f2-8bf6-0f37bca1e333 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.542922 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 954/1906] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:32.548456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.562340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a61cc-89c0-49bf-af3f-b45820f99c12 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.567859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.568621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.568816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.569104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] PATH_INFO: `/v3/projects/c6142fd3c5774a73b6d6e1ab7e21e51a/users/4c044afd99e441b2be3f0169e5a410de/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.586748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6142fd3c5774a73b6d6e1ab7e21e51a, user_id=4c044afd99e441b2be3f0169e5a410de, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.589427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.613534 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 953/1907] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:32 2022] PUT /identity/v3/projects/c6142fd3c5774a73b6d6e1ab7e21e51a/users/4c044afd99e441b2be3f0169e5a410de/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:32.623695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.623966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.624228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.665410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] MFA Rules not processed for user `4c044afd99e441b2be3f0169e5a410de`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:32.666927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:32.672868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:32.808086 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 955/1908] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:32.809987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.910487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7360d-07fb-4601-b68f-c5e5c4a59743 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:32.915511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:32.916128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:32.916401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:32.916666 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:32.917309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:32.919237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:32.926577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:32.959102 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 954/1909] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:32.963494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:32.978695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c357628-b933-4f0c-9b45-ce41a630e874 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.053105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.053891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.054281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.054642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.055452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.057444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.089262 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 956/1910] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:32 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:33.094149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.105837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46337af4-ec63-43e4-a3b9-720fdf86c90c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.112873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.113638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.113977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.114311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.117972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.117972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.129409 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 955/1911] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:33.134161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.146467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8189249-6426-412d-9599-e9acdbb76889 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.153637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d9e00-36fe-46e0-abf1-bc3ef129ea4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.154410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d9e00-36fe-46e0-abf1-bc3ef129ea4d admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.154936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d9e00-36fe-46e0-abf1-bc3ef129ea4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.155344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d9e00-36fe-46e0-abf1-bc3ef129ea4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.156111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d9e00-36fe-46e0-abf1-bc3ef129ea4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.157964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d9e00-36fe-46e0-abf1-bc3ef129ea4d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.176505 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 957/1912] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:33.180855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.191817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae17d89-c9e2-4356-b38c-cbfb734b9471 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.198111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.198961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.199493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.200035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] PATH_INFO: `/v3/projects/3b3b49f8ebcd4ec6af3b5f981f74c3b8/users/579c1f7c348e49f6a8a6d0b00ba57ac4/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.219044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3b49f8ebcd4ec6af3b5f981f74c3b8, user_id=579c1f7c348e49f6a8a6d0b00ba57ac4, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.221988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.247259 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 956/1913] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:33 2022] PUT /identity/v3/projects/3b3b49f8ebcd4ec6af3b5f981f74c3b8/users/579c1f7c348e49f6a8a6d0b00ba57ac4/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:33.251367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.251760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.252080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.320538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] MFA Rules not processed for user `579c1f7c348e49f6a8a6d0b00ba57ac4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:33.320538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:33.457115 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 958/1914] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:33.461705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.580300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27e74a-d035-42e6-bbde-c41fd9a7606b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.590406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.591810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.592088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.592500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.593192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.595939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.655554 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 957/1915] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:33 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:33.662923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.682027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a710a55-120e-4afd-8539-2529ba21bd59 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.700806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:33.797605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.798244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.798519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.798778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.799810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.802012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.863234 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 959/1916] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:33 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:33.865555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.881619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4a239-d39c-485c-a615-60b12959adea admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.887479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.888451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.888900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.889346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.890349 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.892276 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.906786 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 958/1917] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:33.911158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.922379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba96085-6c62-4ba7-98b7-3bb6918115b8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.927728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:33.929289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:33.929289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:33.929496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] PATH_INFO: `/v3/projects/3f5450c301a94cc28b0f885ba64d9bd4/users/c03ffdb63f2e4f40ac7492c8d5115f8a/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:33.962548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5450c301a94cc28b0f885ba64d9bd4, user_id=c03ffdb63f2e4f40ac7492c8d5115f8a, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:33.965000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:33.966221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:33.967200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:33.982151 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cb085-43b4-44bc-966b-0bcd71e9d8e1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:33.988151 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 960/1918] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:33 2022] PUT /identity/v3/projects/3f5450c301a94cc28b0f885ba64d9bd4/users/c03ffdb63f2e4f40ac7492c8d5115f8a/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:33.992225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:34.093160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:34.094134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.094556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.094911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] PATH_INFO: `/v3/users/dcb78d686f2645f589baa7444d88bb30` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.096413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcb78d686f2645f589baa7444d88bb30)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:34.102735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:34.105085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cd475-01ee-4b65-9ba0-fadb90b1261a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:34.119388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:34.121012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.121358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.121741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.122554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:34.128668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:34.145899 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 961/1919] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:34.150526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:34.163133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:34.174440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57476e3a-8ed5-4780-8b60-4bd39b174eb7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:34.179509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7772d9-f821-4a7a-90d3-0c7121985826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:34.180566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7772d9-f821-4a7a-90d3-0c7121985826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.180885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7772d9-f821-4a7a-90d3-0c7121985826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.181156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7772d9-f821-4a7a-90d3-0c7121985826 admin admin] PATH_INFO: `/v3/projects/3f5450c301a94cc28b0f885ba64d9bd4/users/c03ffdb63f2e4f40ac7492c8d5115f8a/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.184153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7772d9-f821-4a7a-90d3-0c7121985826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5450c301a94cc28b0f885ba64d9bd4, user_id=c03ffdb63f2e4f40ac7492c8d5115f8a, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:34.187487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7772d9-f821-4a7a-90d3-0c7121985826 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:34.202047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcb78d686f2645f589baa7444d88bb30'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:34.205554 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 962/1920] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:34 2022] PUT /identity/v3/projects/3f5450c301a94cc28b0f885ba64d9bd4/users/c03ffdb63f2e4f40ac7492c8d5115f8a/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:34.220221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.220613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.220839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.231002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcb78d686f2645f589baa7444d88bb30'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:34.264410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] MFA Rules not processed for user `c03ffdb63f2e4f40ac7492c8d5115f8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:34.267136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:34.281816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcb78d686f2645f589baa7444d88bb30'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:34.282952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcb78d686f2645f589baa7444d88bb30'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:34.292068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-173188bd-6556-4a87-b711-bba57534e233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcb78d686f2645f589baa7444d88bb30'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:34.315004 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 959/1921] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:33 2022] DELETE /identity/v3/users/dcb78d686f2645f589baa7444d88bb30 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:34.319450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.319848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.320215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.347833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:34.379131 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 963/1922] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:34.396663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:34.401928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:34.415773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:34.421441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:34.516292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0f966-2291-4262-9861-54d5dbc64d05 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:34.564569 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 960/1923] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:34.583794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:34.585356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:34.655711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:34.656541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.656941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.657678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.660266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:34.677022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c6bc4-d494-422c-ba16-80ac8d9e7d42 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:34.681932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:34.683604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:34.684055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:34.684332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:34.702476 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:34.704244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:34.717726 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 961/1924] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:34.795485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:34.797203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:34.815383 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 964/1925] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:35.886948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:35.888693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:35.907362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a8fba-b760-4161-b3b0-6aa001ce861f service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:35.913792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:35.914542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:35.914892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:35.915277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] PATH_INFO: `/v3/projects/5fcd185e7a4045e3912adcbdf7646f5e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:35.916583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fcd185e7a4045e3912adcbdf7646f5e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:35.918693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:35.982113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:35.991290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fcd185e7a4045e3912adcbdf7646f5e', 'request_id': 'req-75726c37-17c4-4d24-a1e3-d7503bb29370'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.009676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fcd185e7a4045e3912adcbdf7646f5e', 'request_id': 'req-75726c37-17c4-4d24-a1e3-d7503bb29370'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.018313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fcd185e7a4045e3912adcbdf7646f5e', 'request_id': 'req-75726c37-17c4-4d24-a1e3-d7503bb29370'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.035756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] The token cache is being invalidate because project 5fcd185e7a4045e3912adcbdf7646f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:36.036030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.036709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.043247 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 962/1926] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:35 2022] DELETE /identity/v3/projects/5fcd185e7a4045e3912adcbdf7646f5e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:36.044845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:36.060519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f14e3-c832-4db0-8b98-9a95394ff1e7 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:36.182129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:36.191668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:36.192654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:36.193018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:36.193887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] PATH_INFO: `/v3/users/ce07423d6dd04b6391b81ac748f45429` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:36.195390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce07423d6dd04b6391b81ac748f45429)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:36.197742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:36.261922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce07423d6dd04b6391b81ac748f45429'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.283314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce07423d6dd04b6391b81ac748f45429'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.317248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce07423d6dd04b6391b81ac748f45429'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.318501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce07423d6dd04b6391b81ac748f45429'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.334007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce07423d6dd04b6391b81ac748f45429'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:36.341666 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 965/1927] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:36 2022] DELETE /identity/v3/users/ce07423d6dd04b6391b81ac748f45429 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:36.515597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:36.517873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:36.603213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726c37-17c4-4d24-a1e3-d7503bb29370 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:36.696992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:36.697617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:36.697925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:36.698149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:36.699960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:36.777206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:36.779204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:36.805433 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 963/1928] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:36.846314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:36.847916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:36.981741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe120dc-ee18-4d7b-ab74-f009ed2ce3a6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:36.990446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:36.991103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:36.991481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:36.991748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] PATH_INFO: `/v3/projects/372f508af07a4704b4619c68ead6cccc` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:36.993063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] RBAC: Authorizing `identity:delete_project(project_id=372f508af07a4704b4619c68ead6cccc)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:36.995027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:37.057355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '372f508af07a4704b4619c68ead6cccc', 'request_id': 'req-d5df2936-b73d-49c2-8fa4-3981a8dc786c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.067565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '372f508af07a4704b4619c68ead6cccc', 'request_id': 'req-d5df2936-b73d-49c2-8fa4-3981a8dc786c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.082899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '372f508af07a4704b4619c68ead6cccc', 'request_id': 'req-d5df2936-b73d-49c2-8fa4-3981a8dc786c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.094955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] The token cache is being invalidate because project 372f508af07a4704b4619c68ead6cccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:37.095386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.096195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.098000 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 966/1929] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:36 2022] DELETE /identity/v3/projects/372f508af07a4704b4619c68ead6cccc => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:37.101822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:37.117030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314db884-c70d-416a-bd3e-b76dde07e1ba service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:37.175467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:37.211982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:37.213141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:37.213596 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:37.214031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] PATH_INFO: `/v3/users/5fb32ad613d84efcbd63b2545ab7a40e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:37.216041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb32ad613d84efcbd63b2545ab7a40e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:37.219642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:37.302657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb32ad613d84efcbd63b2545ab7a40e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.317216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb32ad613d84efcbd63b2545ab7a40e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.340463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb32ad613d84efcbd63b2545ab7a40e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.343204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb32ad613d84efcbd63b2545ab7a40e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.355958 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb32ad613d84efcbd63b2545ab7a40e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:37.365118 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 964/1930] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:37 2022] DELETE /identity/v3/users/5fb32ad613d84efcbd63b2545ab7a40e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:37.535001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:37.536654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:37.548379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:37.629878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df2936-b73d-49c2-8fa4-3981a8dc786c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:37.710386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:37.711296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:37.711638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:37.711901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:37.713855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:37.785091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:37.787433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:37.797303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:37.799523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba3855-a1b6-4f31-b325-4dba94d83571 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:37.814417 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 967/1931] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:37.936548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ac56b-263b-461e-8094-8dc2c1db412f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:37.942424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:37.943511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:37.943995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:37.944464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] PATH_INFO: `/v3/projects/877fd038a34a468e90e6dcd5b28205c2` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:37.945909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=877fd038a34a468e90e6dcd5b28205c2)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:37.948728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.006614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877fd038a34a468e90e6dcd5b28205c2', 'request_id': 'req-457577df-845b-47f9-8a7b-12969b89f96d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:38.015895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877fd038a34a468e90e6dcd5b28205c2', 'request_id': 'req-457577df-845b-47f9-8a7b-12969b89f96d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:38.022382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:38.024302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877fd038a34a468e90e6dcd5b28205c2', 'request_id': 'req-457577df-845b-47f9-8a7b-12969b89f96d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:38.051111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] The token cache is being invalidate because project 877fd038a34a468e90e6dcd5b28205c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:38.051519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:38.052296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-457577df-845b-47f9-8a7b-12969b89f96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:38.054726 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 965/1932] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:37 2022] DELETE /identity/v3/projects/877fd038a34a468e90e6dcd5b28205c2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:38.061092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.061971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.063333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.105163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:38.106563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:38.229765 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 968/1933] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:38.240588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.241069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.241246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.267824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:38.269118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:38.277426 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 966/1934] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:38.282671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.295007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e35b9-6516-40b9-9f94-890f5c2d0deb None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.299916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.300534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.300801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.301069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.301703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:38.301996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.303644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.312819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:38.320435 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 969/1935] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:38.325180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.338863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e996ec-420c-4028-921e-eb3322567ca8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.343933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.344669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.345014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.345351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.346052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.348068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.375899 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 967/1936] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:38.379929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.389684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c66cb5-4fe7-4bea-b1de-ad898f9bee80 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.447800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.448429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.448702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.449117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.449783 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.451717 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.474870 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 970/1937] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:38.479280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.489941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23723e-4790-4282-a871-1384d2f1b035 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.494461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.495063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.495371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.495636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.496270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.497848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.507927 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 968/1938] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:38.512016 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.522630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31092dd-dbb3-4386-8b53-a78a24500a5e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.528138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409bbf4c-1d37-49da-b2c3-6b8540eb1e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.528851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bbf4c-1d37-49da-b2c3-6b8540eb1e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.529193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bbf4c-1d37-49da-b2c3-6b8540eb1e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.529527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409bbf4c-1d37-49da-b2c3-6b8540eb1e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.530238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bbf4c-1d37-49da-b2c3-6b8540eb1e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.532036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409bbf4c-1d37-49da-b2c3-6b8540eb1e58 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.543060 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 971/1939] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:38.547874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.558086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a5a6c-971d-4830-96b1-ed927127cd9f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.563549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.564551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.564551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.564902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] PATH_INFO: `/v3/projects/f2e18d7b77df44baaf3e019a5400d1e0/users/837cf96a981d4ba3a18e9f7df7c251ac/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.583717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e18d7b77df44baaf3e019a5400d1e0, user_id=837cf96a981d4ba3a18e9f7df7c251ac, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.586487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.605400 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 969/1940] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:38 2022] PUT /identity/v3/projects/f2e18d7b77df44baaf3e019a5400d1e0/users/837cf96a981d4ba3a18e9f7df7c251ac/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:38.616784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.617066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.617389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.651790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] MFA Rules not processed for user `837cf96a981d4ba3a18e9f7df7c251ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:38.653496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:38.730753 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 972/1941] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:38.735622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.810100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b10676-b1e3-4ee4-b9e5-2f9a26c2c50a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.815856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.816436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.816750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.817045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.817734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.819694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.847224 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 970/1942] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:38.851108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.861896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794a44d-63b9-4adc-98e4-9e0cfa5f5019 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.921780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.922463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.922775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.923045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.923735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.925737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.941880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:38.951486 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 973/1943] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:38 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:38.953587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:38.967351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1e078-a347-41a7-9ecd-691b0ae61152 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:38.973692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:38.974323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:38.974599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:38.975046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:38.975703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:38.977482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:38.991157 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 971/1944] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:38.996053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:39.008722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2377bd-3a8e-49cc-a373-008b57f6c0e3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:39.014406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:39.015115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:39.015397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:39.015673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] PATH_INFO: `/v3/projects/14aca2a21c2246e1aa1d7eaf2f56e956/users/5c1415a9333c41149aafc7f4f2826401/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:39.033068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=14aca2a21c2246e1aa1d7eaf2f56e956, user_id=5c1415a9333c41149aafc7f4f2826401, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:39.035921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:39.052237 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 974/1945] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:38 2022] PUT /identity/v3/projects/14aca2a21c2246e1aa1d7eaf2f56e956/users/5c1415a9333c41149aafc7f4f2826401/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:39.056275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:39.070354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5b4f9-b79b-4749-9a7a-876bb6079a5f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:39.129376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6965e-33be-46e7-936b-4605e5a97a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:39.130294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6965e-33be-46e7-936b-4605e5a97a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:39.130565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6965e-33be-46e7-936b-4605e5a97a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:39.130835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6965e-33be-46e7-936b-4605e5a97a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:39.131585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6965e-33be-46e7-936b-4605e5a97a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:39.133170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6965e-33be-46e7-936b-4605e5a97a8f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:39.135220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:39.144767 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 972/1946] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:39.149870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:39.162602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f16c3-292c-48d6-80be-e41507282dbf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:39.169096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:39.170263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:39.170534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:39.170824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] PATH_INFO: `/v3/projects/14aca2a21c2246e1aa1d7eaf2f56e956/users/5c1415a9333c41149aafc7f4f2826401/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:39.172891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=14aca2a21c2246e1aa1d7eaf2f56e956, user_id=5c1415a9333c41149aafc7f4f2826401, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:39.175611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:39.199259 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 975/1947] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:39 2022] PUT /identity/v3/projects/14aca2a21c2246e1aa1d7eaf2f56e956/users/5c1415a9333c41149aafc7f4f2826401/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:39.203108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:39.203422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:39.203748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:39.232802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] MFA Rules not processed for user `5c1415a9333c41149aafc7f4f2826401`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:39.234038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:39.318107 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 973/1948] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:39.373962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:39.376098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:39.453410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a42c6e-cf50-4cdb-8c7d-c18c7dbc6ccb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:39.458127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6efaf8-deb2-4d8e-b1a2-e1a55000913a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:39.458741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6efaf8-deb2-4d8e-b1a2-e1a55000913a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:39.459149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6efaf8-deb2-4d8e-b1a2-e1a55000913a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:39.459457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6efaf8-deb2-4d8e-b1a2-e1a55000913a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:39.521162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6efaf8-deb2-4d8e-b1a2-e1a55000913a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:39.522605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6efaf8-deb2-4d8e-b1a2-e1a55000913a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:39.534611 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 976/1949] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:41.945681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:41.947540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:42.091725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349cb7f-08ba-4fa7-9efd-7967eb4c34a5 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:42.097387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:42.098480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:42.098957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:42.099494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] PATH_INFO: `/v3/users/4c044afd99e441b2be3f0169e5a410de` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:42.101959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c044afd99e441b2be3f0169e5a410de)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:42.104171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:42.178443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c044afd99e441b2be3f0169e5a410de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:42.190032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c044afd99e441b2be3f0169e5a410de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:42.207128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c044afd99e441b2be3f0169e5a410de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:42.207883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c044afd99e441b2be3f0169e5a410de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:42.215149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c044afd99e441b2be3f0169e5a410de'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:42.221346 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 974/1950] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:41 2022] DELETE /identity/v3/users/4c044afd99e441b2be3f0169e5a410de => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:42.225986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:42.226849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:42.227348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:42.250306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:42.251654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:42.315527 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 977/1951] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:42.326034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:42.327728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:42.333528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:42.396389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359a3da-7ecd-42eb-a257-5a46857debf1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:42.400553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:42.401056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:42.401302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:42.401570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:42.412316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:42.413753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:42.424836 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 975/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:43.076074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:43.077649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:43.093828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b8f4c-b931-430b-8e7d-74c3e717ac02 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:43.104086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:43.105150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:43.106079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:43.106461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] PATH_INFO: `/v3/projects/c6142fd3c5774a73b6d6e1ab7e21e51a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:43.107880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6142fd3c5774a73b6d6e1ab7e21e51a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:43.110240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:43.122057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:43.192664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6142fd3c5774a73b6d6e1ab7e21e51a', 'request_id': 'req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.207743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6142fd3c5774a73b6d6e1ab7e21e51a', 'request_id': 'req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.215313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6142fd3c5774a73b6d6e1ab7e21e51a', 'request_id': 'req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.229151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] The token cache is being invalidate because project c6142fd3c5774a73b6d6e1ab7e21e51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:43.229860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.231288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.236957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:43.239259 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 978/1953] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:43 2022] DELETE /identity/v3/projects/c6142fd3c5774a73b6d6e1ab7e21e51a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:43.251864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d73a6d-3001-4b12-9d2d-3305371fa267 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:43.324984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:43.325800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:43.326077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:43.326359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] PATH_INFO: `/v3/users/579c1f7c348e49f6a8a6d0b00ba57ac4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:43.327551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] RBAC: Authorizing `identity:delete_user(user_id=579c1f7c348e49f6a8a6d0b00ba57ac4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:43.329527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:43.377215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579c1f7c348e49f6a8a6d0b00ba57ac4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.384394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579c1f7c348e49f6a8a6d0b00ba57ac4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.397011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579c1f7c348e49f6a8a6d0b00ba57ac4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.397732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579c1f7c348e49f6a8a6d0b00ba57ac4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.404586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579c1f7c348e49f6a8a6d0b00ba57ac4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.410573 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 976/1954] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:43 2022] DELETE /identity/v3/users/579c1f7c348e49f6a8a6d0b00ba57ac4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:43.683508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:43.684964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:43.701268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b492c5c-edb2-4ad2-a3a0-66d4e8985869 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:43.760868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:43.761541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:43.761815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:43.762085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] PATH_INFO: `/v3/projects/3b3b49f8ebcd4ec6af3b5f981f74c3b8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:43.763256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3b49f8ebcd4ec6af3b5f981f74c3b8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:43.765166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:43.813859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3b49f8ebcd4ec6af3b5f981f74c3b8', 'request_id': 'req-9692e21b-bb93-431b-8ec3-e11d9f20c74e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.824179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3b49f8ebcd4ec6af3b5f981f74c3b8', 'request_id': 'req-9692e21b-bb93-431b-8ec3-e11d9f20c74e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.831399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3b49f8ebcd4ec6af3b5f981f74c3b8', 'request_id': 'req-9692e21b-bb93-431b-8ec3-e11d9f20c74e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.843085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] The token cache is being invalidate because project 3b3b49f8ebcd4ec6af3b5f981f74c3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:43.843488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.844928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:43.846913 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 979/1955] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:43 2022] DELETE /identity/v3/projects/3b3b49f8ebcd4ec6af3b5f981f74c3b8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:43.851669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:43.853421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:43.868522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640c3e2-82bc-4730-ac79-f37b51b0d70e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:43.889002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:43.937043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:43.938257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:43.940578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:43.940578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] PATH_INFO: `/v3/users/c03ffdb63f2e4f40ac7492c8d5115f8a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:43.944010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03ffdb63f2e4f40ac7492c8d5115f8a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:43.948298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:44.001209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03ffdb63f2e4f40ac7492c8d5115f8a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.011458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03ffdb63f2e4f40ac7492c8d5115f8a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.024081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03ffdb63f2e4f40ac7492c8d5115f8a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.027077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03ffdb63f2e4f40ac7492c8d5115f8a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.035543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03ffdb63f2e4f40ac7492c8d5115f8a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.042387 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 977/1956] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:43 2022] DELETE /identity/v3/users/c03ffdb63f2e4f40ac7492c8d5115f8a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:44.295558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:44.297032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:44.330537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:44.362724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692e21b-bb93-431b-8ec3-e11d9f20c74e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:44.374389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:44.377394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:44.435824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:44.436730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:44.437271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:44.437970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:44.441305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:44.487357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bf685-c8da-4354-b2c8-719d64e0ef4d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:44.492621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:44.493799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:44.493931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:44.494198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] PATH_INFO: `/v3/projects/3f5450c301a94cc28b0f885ba64d9bd4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:44.496029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5450c301a94cc28b0f885ba64d9bd4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:44.498308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:44.552378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:44.554131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69524105-b0aa-4621-bb81-80700c9c1c8c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:44.562380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:44.565524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5450c301a94cc28b0f885ba64d9bd4', 'request_id': 'req-047050b8-dab7-433f-83f7-df6114d2e2e1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.567569 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 980/1957] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:44.586805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5450c301a94cc28b0f885ba64d9bd4', 'request_id': 'req-047050b8-dab7-433f-83f7-df6114d2e2e1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.595067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5450c301a94cc28b0f885ba64d9bd4', 'request_id': 'req-047050b8-dab7-433f-83f7-df6114d2e2e1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.609982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] The token cache is being invalidate because project 3f5450c301a94cc28b0f885ba64d9bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:44.610367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.611333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-047050b8-dab7-433f-83f7-df6114d2e2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:44.613134 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 978/1958] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:44 2022] DELETE /identity/v3/projects/3f5450c301a94cc28b0f885ba64d9bd4 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:44.618360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:44.618931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:44.619234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:44.654514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:44.656501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:44.737464 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 981/1959] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:44.745566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:44.746026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:44.746175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:44.770056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:44.771536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:44.784219 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 979/1960] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:44.785895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:44.798621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2f2de-dfe2-4e54-9fbe-a8f662d039a7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:44.804617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:44.805477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:44.805852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:44.806220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:44.807566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:44.807996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:44.809848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:44.825176 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 982/1961] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:44.829717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:44.843386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b889b-2ee9-475b-bd95-2e84cc213452 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:44.848910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:44.849729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:44.850095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:44.850445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:44.851214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:44.853350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:44.892477 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 980/1962] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:44 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:44.912542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:44.927847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a460-5466-4e48-912d-569472b4702a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:44.952490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:45.011889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.012532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.012803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.013078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.013731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.015991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.050419 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 983/1963] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:44 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:45.055227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.072242 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08dbc3-ca12-41f9-82f2-cf98d37eae00 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.077703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.078340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.078690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.079107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.079807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.081795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.094819 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 981/1964] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:45.098757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.111606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46108d2e-d470-49ec-81a3-e164136a09e2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.119768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94474e-ad5a-4f0a-bda4-68b7c339dcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.120844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94474e-ad5a-4f0a-bda4-68b7c339dcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.121281 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94474e-ad5a-4f0a-bda4-68b7c339dcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.121815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94474e-ad5a-4f0a-bda4-68b7c339dcd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.122849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94474e-ad5a-4f0a-bda4-68b7c339dcd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.125536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc94474e-ad5a-4f0a-bda4-68b7c339dcd2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.147482 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 984/1965] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:45.151160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.168557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f58834-41b7-4692-a0dd-9a33be12d03d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.193268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.194091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.194286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.195612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] PATH_INFO: `/v3/projects/0504ef7f072c44f380843d34510aa5f8/users/190ee8efabe046e78050260f9767a433/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.219692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0504ef7f072c44f380843d34510aa5f8, user_id=190ee8efabe046e78050260f9767a433, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.222793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.244475 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 982/1966] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:45 2022] PUT /identity/v3/projects/0504ef7f072c44f380843d34510aa5f8/users/190ee8efabe046e78050260f9767a433/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:45.257222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.257683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.258080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.292946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] MFA Rules not processed for user `190ee8efabe046e78050260f9767a433`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:45.294889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:45.371067 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 985/1967] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:45.375519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.444635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b7586-a100-43f9-ba82-d7702a437de5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.449573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.450187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.450458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.450723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.451401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.453293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.491637 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 983/1968] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:45 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:45.496356 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.508736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7518fe-4cf3-4f9a-b0d7-a4b534912b42 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.567615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.568223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.568493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.568759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.569379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.571309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.597350 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 986/1969] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:45 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:45.598873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.610875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941fc32-7c9a-4fdd-a676-10458e0cc685 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.617319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.618143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.618419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.618681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.619357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.621265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.632322 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 984/1970] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:45.636620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.650876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45986928-0cf4-458c-b68c-c10aacafade1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.656383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc23a9-f67e-4c6a-8065-f5064eca8d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.657203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc23a9-f67e-4c6a-8065-f5064eca8d75 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.657582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc23a9-f67e-4c6a-8065-f5064eca8d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.657963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc23a9-f67e-4c6a-8065-f5064eca8d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.658808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc23a9-f67e-4c6a-8065-f5064eca8d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.660518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc23a9-f67e-4c6a-8065-f5064eca8d75 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.673602 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 987/1971] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:45.678246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.688712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf761f69-ceb2-4533-a98f-7d7c8b8c2aae admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.693503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.694162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.694428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.694686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] PATH_INFO: `/v3/projects/78a76257c258414797f5119bfe7fcd52/users/192987cb9e47417e89c520772e894647/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.709934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a76257c258414797f5119bfe7fcd52, user_id=192987cb9e47417e89c520772e894647, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.712617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.727726 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 985/1972] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:45 2022] PUT /identity/v3/projects/78a76257c258414797f5119bfe7fcd52/users/192987cb9e47417e89c520772e894647/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:45.738258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.738643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.738983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.760385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:45.769987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] MFA Rules not processed for user `192987cb9e47417e89c520772e894647`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:45.771643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:45.837930 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 988/1973] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:45.843141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.870483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:45.916266 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4953b-529b-4ea6-92a2-b0166ad2c3a4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:45.921804 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:45.922522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:45.922857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:45.923121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:45.923778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:45.925660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:45.954260 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 986/1974] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:45 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:45.958509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:45.970233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef953c-ee7b-465d-b7be-4c4c7a3a2468 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:46.029066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:46.029672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.029952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.030217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.030853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:46.032885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:46.058603 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 989/1975] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:45 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:46.062818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:46.073385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1954e-d653-4b5f-aa74-3c66a85d0fd7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:46.078246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:46.078890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.079164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.079469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.080143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:46.081890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:46.097585 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 987/1976] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:46.097873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:46.108812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467dbca-0b9a-4aaf-87b6-a8bd82f3631c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:46.114857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:46.115558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.115835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.116107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] PATH_INFO: `/v3/projects/b9d523ddeb064e828aec7a1eb34abbf7/users/4a2c41a090264ae481ab218217d1af8d/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.132238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d523ddeb064e828aec7a1eb34abbf7, user_id=4a2c41a090264ae481ab218217d1af8d, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:46.134873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:46.151826 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 990/1977] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:46 2022] PUT /identity/v3/projects/b9d523ddeb064e828aec7a1eb34abbf7/users/4a2c41a090264ae481ab218217d1af8d/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:46.155800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:46.167336 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e9a17-b5d7-4478-ac52-3435a8bd7109 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:46.232772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6cc4a-946c-4b0f-8380-34df76360f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:46.233449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6cc4a-946c-4b0f-8380-34df76360f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.233751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6cc4a-946c-4b0f-8380-34df76360f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.234117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6cc4a-946c-4b0f-8380-34df76360f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.234992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6cc4a-946c-4b0f-8380-34df76360f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:46.236691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f6cc4a-946c-4b0f-8380-34df76360f68 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:46.251513 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 988/1978] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:46.253255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:46.266126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dca039-be16-4dba-8889-5c0b278fd626 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:46.270669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:46.271381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.271652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.271914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] PATH_INFO: `/v3/projects/b9d523ddeb064e828aec7a1eb34abbf7/users/4a2c41a090264ae481ab218217d1af8d/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.274339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d523ddeb064e828aec7a1eb34abbf7, user_id=4a2c41a090264ae481ab218217d1af8d, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:46.277304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:46.294792 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 991/1979] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:46 2022] PUT /identity/v3/projects/b9d523ddeb064e828aec7a1eb34abbf7/users/4a2c41a090264ae481ab218217d1af8d/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:46.299778 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.299978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.300254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.329459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] MFA Rules not processed for user `4a2c41a090264ae481ab218217d1af8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:46.330662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:46.401096 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 989/1980] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:46.413883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:46.415729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:46.472995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d5cc-90a0-47ec-8650-d7f21bf6f46e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:46.476534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:46.477131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:46.477399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:46.477669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:46.481700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:46.540808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:46.542241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:46.552576 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 992/1981] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:49.156468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:49.167574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72565ad5-36ef-4ae0-8e4f-8fdbe047564b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:49.171830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:49.172445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:49.172715 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:49.172970 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:49.187534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:49.234746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:49.236274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:49.245636 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 990/1982] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:49.847447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:49.856931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229f828-1e82-4730-9b5b-8d9b8afcd212 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:49.861425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22539b0a-d0c3-41d2-8fb8-2122c7c8bda0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:49.862038 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539b0a-d0c3-41d2-8fb8-2122c7c8bda0 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:49.862330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539b0a-d0c3-41d2-8fb8-2122c7c8bda0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:49.862601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22539b0a-d0c3-41d2-8fb8-2122c7c8bda0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:49.872729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539b0a-d0c3-41d2-8fb8-2122c7c8bda0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:49.874165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22539b0a-d0c3-41d2-8fb8-2122c7c8bda0 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:49.886062 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 993/1983] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:50.698985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:50.700597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:50.769278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a45b7-d73d-41aa-8372-926fa404a9b8 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:50.774310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:50.774937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:50.775247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:50.775517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] PATH_INFO: `/v3/users/837cf96a981d4ba3a18e9f7df7c251ac` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:50.776700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837cf96a981d4ba3a18e9f7df7c251ac)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:50.778688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:50.828473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837cf96a981d4ba3a18e9f7df7c251ac'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:50.838449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837cf96a981d4ba3a18e9f7df7c251ac'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:50.853088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837cf96a981d4ba3a18e9f7df7c251ac'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:50.853851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837cf96a981d4ba3a18e9f7df7c251ac'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:50.861649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837cf96a981d4ba3a18e9f7df7c251ac'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:50.867685 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 991/1984] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:50 2022] DELETE /identity/v3/users/837cf96a981d4ba3a18e9f7df7c251ac => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:50.872338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:50.872812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:50.872929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:50.892336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:50.893553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:50.956318 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 994/1985] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:50.966360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:50.968063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:51.003986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:51.032595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdc75d-ccaa-4561-a9bc-cdfcfb674b58 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:51.037163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:51.037781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:51.038055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:51.038325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:51.048740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:51.050361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:51.061127 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 992/1986] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:51.352548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:51.354274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:51.369768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de405a0-cce8-4155-bcf1-9fc6c307314f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:51.380825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:51.381629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:51.381921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:51.382181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] PATH_INFO: `/v3/projects/f2e18d7b77df44baaf3e019a5400d1e0` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:51.383517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e18d7b77df44baaf3e019a5400d1e0)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:51.385402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:51.441862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e18d7b77df44baaf3e019a5400d1e0', 'request_id': 'req-8a76c752-4c2f-4dee-9b2c-13270394751a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.453781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e18d7b77df44baaf3e019a5400d1e0', 'request_id': 'req-8a76c752-4c2f-4dee-9b2c-13270394751a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.458737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e18d7b77df44baaf3e019a5400d1e0', 'request_id': 'req-8a76c752-4c2f-4dee-9b2c-13270394751a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.471409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] The token cache is being invalidate because project f2e18d7b77df44baaf3e019a5400d1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:51.471696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.472355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.473985 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 995/1987] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:51 2022] DELETE /identity/v3/projects/f2e18d7b77df44baaf3e019a5400d1e0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:51.477992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:51.487954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29532116-50fa-4e02-a7c9-2c3e7098849f service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:51.544860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:51.545408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:51.545682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:51.545915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] PATH_INFO: `/v3/users/5c1415a9333c41149aafc7f4f2826401` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:51.547413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c1415a9333c41149aafc7f4f2826401)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:51.549443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:51.603582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c1415a9333c41149aafc7f4f2826401'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.613957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c1415a9333c41149aafc7f4f2826401'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.630210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c1415a9333c41149aafc7f4f2826401'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.631087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c1415a9333c41149aafc7f4f2826401'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.641997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1b23af39-1c57-4a43-a21d-9c1fa0be869c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c1415a9333c41149aafc7f4f2826401'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:51.649611 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 993/1988] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:51 2022] DELETE /identity/v3/users/5c1415a9333c41149aafc7f4f2826401 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:52.205142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.206737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:52.220536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76c752-4c2f-4dee-9b2c-13270394751a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.237845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:52.295126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.295793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.296062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.296326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] PATH_INFO: `/v3/projects/14aca2a21c2246e1aa1d7eaf2f56e956` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.297440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14aca2a21c2246e1aa1d7eaf2f56e956)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.299418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.350264 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14aca2a21c2246e1aa1d7eaf2f56e956', 'request_id': 'req-b58dd6fd-52bb-4398-894a-5cf903866c94'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:52.358490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14aca2a21c2246e1aa1d7eaf2f56e956', 'request_id': 'req-b58dd6fd-52bb-4398-894a-5cf903866c94'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:52.365750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14aca2a21c2246e1aa1d7eaf2f56e956', 'request_id': 'req-b58dd6fd-52bb-4398-894a-5cf903866c94'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:52.380197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] The token cache is being invalidate because project 14aca2a21c2246e1aa1d7eaf2f56e956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:52.380758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:52.381630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b58dd6fd-52bb-4398-894a-5cf903866c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:52.383815 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 996/1989] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:52 2022] DELETE /identity/v3/projects/14aca2a21c2246e1aa1d7eaf2f56e956 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:52.391244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.391244 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.391878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.417494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:52.418787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:52.474253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:52.487224 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 994/1990] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:52.494349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.494627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.494894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.519901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:52.521543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:52.532881 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 997/1991] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:52.534494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.548373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785ad0b-2fa0-47bf-8354-652c67d3a706 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.556024 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.556650 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.556927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.557192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.557848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:52.558149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.559766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.577818 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 995/1992] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:52.579047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.589196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8b56-325d-45af-80c7-909f887f6528 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.594992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.595749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.596099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.596417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.597132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.599206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.639652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.642702 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 998/1993] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:52 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:52.665101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f58855-85a9-4f77-9618-29b3ec160408 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.749471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.750983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.751359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.751650 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.752464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.755303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.792548 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 996/1994] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:52 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:52.794855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.817352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6697a5-97a4-47dd-ae18-a315fa045847 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.831216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.831866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.832172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.832432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.833080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.835056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.864221 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 999/1995] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:52.864403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.874528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c52cf-b7df-4762-9a57-20914117481d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.879317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd4bfc-c912-42a1-b2c8-8697bbda0570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.879922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd4bfc-c912-42a1-b2c8-8697bbda0570 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.880193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd4bfc-c912-42a1-b2c8-8697bbda0570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.880457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd4bfc-c912-42a1-b2c8-8697bbda0570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.881088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd4bfc-c912-42a1-b2c8-8697bbda0570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.882687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd4bfc-c912-42a1-b2c8-8697bbda0570 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.894724 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 997/1996] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:52.899059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:52.909174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549811a-c498-448d-9373-8c032bfdde7e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:52.913786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:52.914540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.914882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.915271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] PATH_INFO: `/v3/projects/02a7e1461ef24da8a1cd7d5cbb50cbae/users/858d7323da1b4b8a9c9c58479f75eed9/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:52.937102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a7e1461ef24da8a1cd7d5cbb50cbae, user_id=858d7323da1b4b8a9c9c58479f75eed9, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:52.940151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:52.960084 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1000/1997] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:52 2022] PUT /identity/v3/projects/02a7e1461ef24da8a1cd7d5cbb50cbae/users/858d7323da1b4b8a9c9c58479f75eed9/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:52.972939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:52.973227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:52.973467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:53.015837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] MFA Rules not processed for user `858d7323da1b4b8a9c9c58479f75eed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:53.018145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:53.085741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:53.089338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:53.127443 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 998/1998] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:53.148328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:53.152676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:53.182569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:53.209516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604560ce-41d8-4967-84cc-8dd16a6a5c07 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:53.215438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:53.216227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:53.216508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:53.216780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] PATH_INFO: `/v3/users/57c2cdd9626146a1881a3d1b04509cde` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:53.218057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57c2cdd9626146a1881a3d1b04509cde)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:53.220009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:53.228987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6211c8b-2d13-4c23-b125-4edfc4024a52 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:53.233066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f41aa7-24b4-41ed-93ba-9ac730378d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:53.233560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f41aa7-24b4-41ed-93ba-9ac730378d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:53.233903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f41aa7-24b4-41ed-93ba-9ac730378d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:53.234238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f41aa7-24b4-41ed-93ba-9ac730378d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:53.241129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:53.246169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f41aa7-24b4-41ed-93ba-9ac730378d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:53.247448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f41aa7-24b4-41ed-93ba-9ac730378d6b service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:53.260036 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 999/1999] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:53.277309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57c2cdd9626146a1881a3d1b04509cde'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:53.290136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57c2cdd9626146a1881a3d1b04509cde'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:53.311769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57c2cdd9626146a1881a3d1b04509cde'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:53.312814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57c2cdd9626146a1881a3d1b04509cde'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:53.329970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57c2cdd9626146a1881a3d1b04509cde'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:53.335996 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1001/2000] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:53 2022] DELETE /identity/v3/users/57c2cdd9626146a1881a3d1b04509cde => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:53.340662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:53.341047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:53.341412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:53.361670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:53.363133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:53.468819 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1000/2001] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:53.480912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:53.483848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:53.571421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f64756-9255-49a5-aff6-586a7b810299 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:53.576200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:53.576929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:53.577469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:53.577822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:53.604466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:53.606448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:53.621630 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1002/2002] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:53.986210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:53.987761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.001068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d831816-9240-4230-a98f-485ef375b4bc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.015382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.016408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.017051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.017553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] PATH_INFO: `/v3/users/190ee8efabe046e78050260f9767a433` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.019035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190ee8efabe046e78050260f9767a433)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.021254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.074298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.076463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.090628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5df827-4733-4a68-9644-e8a816703ed0 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.093524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190ee8efabe046e78050260f9767a433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.095750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.098331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.098617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.098890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] PATH_INFO: `/v3/projects/e063ad95dc8644e8a5ea9c38c75eb990` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.100652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e063ad95dc8644e8a5ea9c38c75eb990)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.105821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.106859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190ee8efabe046e78050260f9767a433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.122488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190ee8efabe046e78050260f9767a433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.123843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190ee8efabe046e78050260f9767a433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.131913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d52acde6-a69e-44ea-9865-195dd62e5523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190ee8efabe046e78050260f9767a433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.145907 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1001/2003] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:53 2022] DELETE /identity/v3/users/190ee8efabe046e78050260f9767a433 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:54.145907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68bf8f-db81-417d-9c78-5a0541e77f25 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.145907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68bf8f-db81-417d-9c78-5a0541e77f25 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.145907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68bf8f-db81-417d-9c78-5a0541e77f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.168373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dd68bf8f-db81-417d-9c78-5a0541e77f25 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:54.170034 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dd68bf8f-db81-417d-9c78-5a0541e77f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.186594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e063ad95dc8644e8a5ea9c38c75eb990', 'request_id': 'req-76290beb-a42a-4fc7-89c6-59562689313a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.199078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e063ad95dc8644e8a5ea9c38c75eb990', 'request_id': 'req-76290beb-a42a-4fc7-89c6-59562689313a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.204538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:54.212821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e063ad95dc8644e8a5ea9c38c75eb990', 'request_id': 'req-76290beb-a42a-4fc7-89c6-59562689313a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.228422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] The token cache is being invalidate because project e063ad95dc8644e8a5ea9c38c75eb990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:54.229102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.229443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-76290beb-a42a-4fc7-89c6-59562689313a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:54.231676 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1003/2004] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:54 2022] DELETE /identity/v3/projects/e063ad95dc8644e8a5ea9c38c75eb990 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:54.241156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.241684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.241948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.263103 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1002/2005] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:54.267881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:54.272418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.285334 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1004/2006] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:54.293338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.293656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.294315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.305259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.306726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.315583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:54.335548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:54.337647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.358016 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1003/2007] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:54.365381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.382795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18260fe-7430-45dd-a0ee-53597432303d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.395756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.398704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.398988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.400387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.401776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:54.402688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.404984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.433676 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1004/2008] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:54.437935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.445866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe54b2-a645-4bc1-a5d1-c0a1658f70a4 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.456638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.457710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.458080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.458499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.460083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b75587-e056-45f2-88cf-94a16147fd01 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.467454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.468368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.468790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.469268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.470850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.476346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.507495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.507495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.517588 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1005/2009] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:54.540560 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1005/2010] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:54 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:54.541659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.554392 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568b165-e7c3-4f9e-b283-74a9970b96d3 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.628024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.628733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.629050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.629357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.630338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.633316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.662452 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1006/2011] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:54 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:54.664541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.690900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434598-8cd7-4c64-a158-afd66d020e41 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.700813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.701927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.702405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.702831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.704981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.708204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.726734 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1006/2012] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:54.730568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.744485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754dc1f-7fef-4f4e-8a12-887c566ce448 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.757051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.760888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.761703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.761975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.771728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.773789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.795918 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1007/2013] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:54.797291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.815662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b128e1-1da9-4bc9-80ea-2905bcf3e614 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.824654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5025f9-7c4c-4dcd-8f2c-4112fb551e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.826871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5025f9-7c4c-4dcd-8f2c-4112fb551e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.827149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5025f9-7c4c-4dcd-8f2c-4112fb551e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.827462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5025f9-7c4c-4dcd-8f2c-4112fb551e12 admin admin] PATH_INFO: `/v3/projects/7ffcdce933124bacba7b5b1f07b8f05d/users/371e2b43dd0e431ca4059a403495c59e/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.861073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5025f9-7c4c-4dcd-8f2c-4112fb551e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ffcdce933124bacba7b5b1f07b8f05d, user_id=371e2b43dd0e431ca4059a403495c59e, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.868369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5025f9-7c4c-4dcd-8f2c-4112fb551e12 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:54.881248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:54.883015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:54.903169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf2c8d-7ef2-4d42-9747-d7d5967cc591 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:54.918332 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1007/2014] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:54 2022] PUT /identity/v3/projects/7ffcdce933124bacba7b5b1f07b8f05d/users/371e2b43dd0e431ca4059a403495c59e/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:54.933082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd22b-9348-4669-a333-120a3269021b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.933082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd22b-9348-4669-a333-120a3269021b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.933314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15dd22b-9348-4669-a333-120a3269021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.937462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:54.938120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:54.938394 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:54.938660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] PATH_INFO: `/v3/projects/0504ef7f072c44f380843d34510aa5f8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:54.939901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0504ef7f072c44f380843d34510aa5f8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:54.946670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:55.010514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a15dd22b-9348-4669-a333-120a3269021b None None] MFA Rules not processed for user `371e2b43dd0e431ca4059a403495c59e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:55.011818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a15dd22b-9348-4669-a333-120a3269021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:55.080039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0504ef7f072c44f380843d34510aa5f8', 'request_id': 'req-7b436391-35a2-4e02-8642-336a97547f76'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.095446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0504ef7f072c44f380843d34510aa5f8', 'request_id': 'req-7b436391-35a2-4e02-8642-336a97547f76'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.099621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:55.133317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0504ef7f072c44f380843d34510aa5f8', 'request_id': 'req-7b436391-35a2-4e02-8642-336a97547f76'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.205241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] The token cache is being invalidate because project 0504ef7f072c44f380843d34510aa5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:55.205583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.213360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.221680 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1008/2015] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:54 2022] DELETE /identity/v3/projects/0504ef7f072c44f380843d34510aa5f8 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:55.226320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:55.230286 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1008/2016] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:55.230749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:55.244033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd22b-9348-4669-a333-120a3269021b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:55.248767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a15dd22b-9348-4669-a333-120a3269021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:55.388802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b436391-35a2-4e02-8642-336a97547f76 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:55.399215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:55.401148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:55.401696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:55.402027 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] PATH_INFO: `/v3/users/192987cb9e47417e89c520772e894647` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:55.403900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192987cb9e47417e89c520772e894647)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:55.406978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:55.434402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15dd22b-9348-4669-a333-120a3269021b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:55.487914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192987cb9e47417e89c520772e894647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.500204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192987cb9e47417e89c520772e894647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.501651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:55.522727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192987cb9e47417e89c520772e894647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.523764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192987cb9e47417e89c520772e894647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.533389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:55.534025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:55.534294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:55.534615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:55.537936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:55.563427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e306263f-63de-431f-bcb2-d60af896d3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192987cb9e47417e89c520772e894647'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:55.572813 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1009/2017] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:55 2022] DELETE /identity/v3/users/192987cb9e47417e89c520772e894647 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:55.665040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:55.668211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437e8c8-a0cb-4aa7-8f93-f491a46d60be service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:55.703222 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1009/2018] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:56:55.834765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43884f80-534a-4aba-a50d-04378d75c235 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:55.835058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43884f80-534a-4aba-a50d-04378d75c235 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:55.835376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43884f80-534a-4aba-a50d-04378d75c235 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:55.837059 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1010/2019] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:56:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:56:55.854207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:55.854874 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:55.856001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:55.905726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] MFA Rules not processed for user `00e3a0e394ba43f3a2362c3ac69a2fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:55.906977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:55.922438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43884f80-534a-4aba-a50d-04378d75c235 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:55.924271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-43884f80-534a-4aba-a50d-04378d75c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.000556 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1010/2020] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:56:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:56.001819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.003352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.025815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43884f80-534a-4aba-a50d-04378d75c235 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.030976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.031582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.031929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.032161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] PATH_INFO: `/v3/users/f06b14789a2740eca22001c0b5232fc4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.033310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06b14789a2740eca22001c0b5232fc4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.035401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.041889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:56.111864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d39ec-519b-4a73-a7d1-1d03d3c39b23 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.112401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06b14789a2740eca22001c0b5232fc4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.117407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.118278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.118561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.118912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] PATH_INFO: `/v3/projects/78a76257c258414797f5119bfe7fcd52` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.120189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a76257c258414797f5119bfe7fcd52)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.122271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.124656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06b14789a2740eca22001c0b5232fc4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.135470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06b14789a2740eca22001c0b5232fc4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.136283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06b14789a2740eca22001c0b5232fc4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.166882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06b14789a2740eca22001c0b5232fc4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.177678 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1011/2021] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:55 2022] DELETE /identity/v3/users/f06b14789a2740eca22001c0b5232fc4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:56.178371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.179766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.185296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:56.210633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a76257c258414797f5119bfe7fcd52', 'request_id': 'req-870af54f-17b3-42c3-b079-a951f3746863'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.223939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a76257c258414797f5119bfe7fcd52', 'request_id': 'req-870af54f-17b3-42c3-b079-a951f3746863'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.234202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a76257c258414797f5119bfe7fcd52', 'request_id': 'req-870af54f-17b3-42c3-b079-a951f3746863'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.262245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] The token cache is being invalidate because project 78a76257c258414797f5119bfe7fcd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:56.262539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.263386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.268860 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1011/2022] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:56 2022] DELETE /identity/v3/projects/78a76257c258414797f5119bfe7fcd52 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:56.269830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.271256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.273725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e92f3-fa33-4e46-95a1-b4ef04eb5b2a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.353000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af54f-17b3-42c3-b079-a951f3746863 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.353292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.354522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.354797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.355062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.357051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.358236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.358964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.359271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.359536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] PATH_INFO: `/v3/users/4a2c41a090264ae481ab218217d1af8d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.360673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2c41a090264ae481ab218217d1af8d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.362686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.371335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.373000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.387418 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1012/2023] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:56.405938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.407738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.424454 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95e433-1f6a-41ef-ade9-221707170890 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.430679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.431305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.431582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.432465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] PATH_INFO: `/v3/projects/a2d58c81f2e94dd79d2fdcf08b260636` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.436197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d58c81f2e94dd79d2fdcf08b260636)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.438620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.443726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2c41a090264ae481ab218217d1af8d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.455865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2c41a090264ae481ab218217d1af8d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.459320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:56.473225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2c41a090264ae481ab218217d1af8d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.474752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2c41a090264ae481ab218217d1af8d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.484376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8cf75af-d959-4514-9a35-83cdf2f20ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2c41a090264ae481ab218217d1af8d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.491859 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1012/2024] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:56:56 2022] DELETE /identity/v3/users/4a2c41a090264ae481ab218217d1af8d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:56.506943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d58c81f2e94dd79d2fdcf08b260636', 'request_id': 'req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.516943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d58c81f2e94dd79d2fdcf08b260636', 'request_id': 'req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.525865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d58c81f2e94dd79d2fdcf08b260636', 'request_id': 'req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.541186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] The token cache is being invalidate because project a2d58c81f2e94dd79d2fdcf08b260636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:56.541477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.542562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d05e95e7-a1c5-4d11-8b6e-142e505ac99c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.545445 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1013/2025] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:56 2022] DELETE /identity/v3/projects/a2d58c81f2e94dd79d2fdcf08b260636 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:56.550488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.551702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.552056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.578888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:56.580752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.625386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:56.676477 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1013/2026] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:56.684811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.685094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.685367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.705818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:56.707050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.716545 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1014/2027] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:56.720914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.731455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cba33e-c2d3-4979-924d-47ff7bc5b362 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.737063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.737680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.737960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.738284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.739072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:56.739405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.741021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.753166 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1014/2028] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:56.757867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.767674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b77d997-ed54-4a7a-a032-594020939b28 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.772816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.773429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.773745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.774011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.774849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.776989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.795421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.797255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:56.812245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51936ceb-aac4-45ab-9a16-c8cd79846950 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.815282 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1015/2029] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:56 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:56.816456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.906273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b76c61-81ee-4d26-97d3-28ab5f9c848d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.911998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.912603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:56.912739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.912939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.913258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.913430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:56.913685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:56.913954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.914064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] PATH_INFO: `/v3/projects/b9d523ddeb064e828aec7a1eb34abbf7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:56.916216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d523ddeb064e828aec7a1eb34abbf7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:56.916344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.919502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:56.950105 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1016/2030] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:56 2022] POST /identity/v3/users => generated 284 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:56:56.951411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:56.977018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ff64b-298a-46be-9022-d3c8498bbbf4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:56.986620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d523ddeb064e828aec7a1eb34abbf7', 'request_id': 'req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:56.997808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d523ddeb064e828aec7a1eb34abbf7', 'request_id': 'req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:57.004077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d523ddeb064e828aec7a1eb34abbf7', 'request_id': 'req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:57.017117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] The token cache is being invalidate because project b9d523ddeb064e828aec7a1eb34abbf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:56:57.017404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:57.018099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c4f5be21-7092-4d3c-aabf-2d2bd0e4f20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:56:57.020041 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1015/2031] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:56:56 2022] DELETE /identity/v3/projects/b9d523ddeb064e828aec7a1eb34abbf7 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:57.025391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08db2e-8e88-40ee-a496-e79d424e710f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.025737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08db2e-8e88-40ee-a496-e79d424e710f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.026115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08db2e-8e88-40ee-a496-e79d424e710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.039986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.040598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.040871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.041145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.041809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.043426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.045060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:57.046456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ea08db2e-8e88-40ee-a496-e79d424e710f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:57.047817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ea08db2e-8e88-40ee-a496-e79d424e710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.055133 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1017/2032] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:57.056896 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1016/2033] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:57.063136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.063136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.064839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.065177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.065447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.074238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f9134-3e43-4457-a28e-e54fb94f51af admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.079148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.079793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.080054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.080316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.080961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.082609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.086327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:57.087569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.093402 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1018/2034] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:56:57.096202 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1017/2035] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:56:57.097056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.100245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.107089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f4a6a2-e80d-48e6-822e-841df68f6b78 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.109227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468024fc-71a8-4744-958d-c767c6b6ee8d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.112171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fcd7c-bf44-4218-b619-e601dbdfc310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.112964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fcd7c-bf44-4218-b619-e601dbdfc310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.113278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fcd7c-bf44-4218-b619-e601dbdfc310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.113608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fcd7c-bf44-4218-b619-e601dbdfc310 admin admin] PATH_INFO: `/v3/projects/b72e36a56afe46bc94dde799bae91a89/users/6d1ab82fe4fe41e391b27a4847449de4/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.114081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.114643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.114913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.115190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.115836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:56:57.116125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.117692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.129747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fcd7c-bf44-4218-b619-e601dbdfc310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72e36a56afe46bc94dde799bae91a89, user_id=6d1ab82fe4fe41e391b27a4847449de4, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.130111 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1018/2036] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:56:57.132448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fcd7c-bf44-4218-b619-e601dbdfc310 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.134327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.144196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262f37d-2007-4330-b9be-c62473be53b6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.150730 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1019/2037] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:57 2022] PUT /identity/v3/projects/b72e36a56afe46bc94dde799bae91a89/users/6d1ab82fe4fe41e391b27a4847449de4/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:56:57.161311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.161589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.161863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.180436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:57.201750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] MFA Rules not processed for user `6d1ab82fe4fe41e391b27a4847449de4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:57.203122 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.219998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.220723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.221066 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.221429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.222286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.225319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.256836 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1019/2038] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:56:57 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:57.261272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.281431 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1020/2039] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:57.294517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.295781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.336435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2da33-792c-4d8f-9249-bfe11ab1d0ca admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.343425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.344042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.344316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.344585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.345199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.347105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.359337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76c1f0-e125-4a02-b091-bd29cb9bbafc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.373951 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1020/2040] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:56:57 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:56:57.375091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.388443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b058b-2b5f-4dc2-9881-963049564a3e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.396041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.396839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.397262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.397651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.398492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.400607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.413451 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1021/2041] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:57.417380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.422798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aca111-0bde-427a-8169-a13dba4b026a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.423373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca111-0bde-427a-8169-a13dba4b026a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.423644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca111-0bde-427a-8169-a13dba4b026a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.423912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aca111-0bde-427a-8169-a13dba4b026a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.428202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1beb4bc-f2fb-4dd9-9fb0-c5accb58e921 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.433949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.434764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.435206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.435490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.436132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.437734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.443417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:57.453358 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1022/2042] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:56:57.454494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.464356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1193fa-a8ab-4cf2-86fe-ecc55eb7a205 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.470216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.470864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.471141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.471437 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] PATH_INFO: `/v3/projects/a84206f4832a4edda41c06ab2178c9a5/users/398da9e11a194b9f9644dfad517d07b4/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.484380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aca111-0bde-427a-8169-a13dba4b026a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.485842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aca111-0bde-427a-8169-a13dba4b026a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.486946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84206f4832a4edda41c06ab2178c9a5, user_id=398da9e11a194b9f9644dfad517d07b4, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.489713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.495168 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1021/2043] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:56:57.503115 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1023/2044] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:56:57 2022] PUT /identity/v3/projects/a84206f4832a4edda41c06ab2178c9a5/users/398da9e11a194b9f9644dfad517d07b4/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:56:57.508497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef097d6-7fc6-48df-a21a-bf9e9cd9292f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.508770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef097d6-7fc6-48df-a21a-bf9e9cd9292f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.509036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef097d6-7fc6-48df-a21a-bf9e9cd9292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.528063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5ef097d6-7fc6-48df-a21a-bf9e9cd9292f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:57.529247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5ef097d6-7fc6-48df-a21a-bf9e9cd9292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.594259 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1022/2045] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:57.604795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:57.606865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:57.676198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaebca96-af07-4503-8fa5-9a558010d71c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:57.680671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:57.681306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:57.681552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:57.681805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:57.695309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:57.696841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:57.701464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:56:57.708169 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1024/2046] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:56:58.471000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf14d3c-ded0-48f7-90bb-4513eeb697a3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:58.471560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf14d3c-ded0-48f7-90bb-4513eeb697a3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:58.471909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf14d3c-ded0-48f7-90bb-4513eeb697a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:58.504280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4bf14d3c-ded0-48f7-90bb-4513eeb697a3 None None] MFA Rules not processed for user `398da9e11a194b9f9644dfad517d07b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:56:58.505695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4bf14d3c-ded0-48f7-90bb-4513eeb697a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:58.591707 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1023/2047] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:56:58.601280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:56:58.603415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:56:58.674692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e53f30-c19d-4b6c-85a2-7661ada0fe2c service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:56:58.678414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:56:58.679014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:56:58.679321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:56:58.679585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:56:58.688797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:56:58.690252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:56:58.700221 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1025/2048] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:56:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:00.187940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb43499-6814-47b9-90da-fcc6099ee6ac None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:00.188539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb43499-6814-47b9-90da-fcc6099ee6ac None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:00.188539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb43499-6814-47b9-90da-fcc6099ee6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:00.208240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-adb43499-6814-47b9-90da-fcc6099ee6ac None None] MFA Rules not processed for user `398da9e11a194b9f9644dfad517d07b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:00.209456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-adb43499-6814-47b9-90da-fcc6099ee6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:00.216985 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1024/2049] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:02.446137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:02.447800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:02.463120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0211c3-50eb-4294-bf6b-b985bcd1f981 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:02.468228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:02.468861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:02.469170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:02.469443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] PATH_INFO: `/v3/users/371e2b43dd0e431ca4059a403495c59e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:02.470674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371e2b43dd0e431ca4059a403495c59e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:02.472897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:02.579968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371e2b43dd0e431ca4059a403495c59e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:02.590537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371e2b43dd0e431ca4059a403495c59e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:02.611160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371e2b43dd0e431ca4059a403495c59e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:02.612878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371e2b43dd0e431ca4059a403495c59e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:02.628717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371e2b43dd0e431ca4059a403495c59e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:02.637636 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1026/2050] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:02 2022] DELETE /identity/v3/users/371e2b43dd0e431ca4059a403495c59e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:02.641816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:02.642336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:02.642336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:02.666677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:02.668667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:02.731738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:02.785724 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1025/2051] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:57:02.797779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:02.799318 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:02.879507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248d6d1-9027-4c11-af9f-cdd9bb58b887 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:02.884382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3076e-8a47-40fb-9aa8-e1a8e48ba717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:02.885158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3076e-8a47-40fb-9aa8-e1a8e48ba717 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:02.885543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3076e-8a47-40fb-9aa8-e1a8e48ba717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:02.885916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3076e-8a47-40fb-9aa8-e1a8e48ba717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:02.900924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3076e-8a47-40fb-9aa8-e1a8e48ba717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:02.902437 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3076e-8a47-40fb-9aa8-e1a8e48ba717 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:02.917435 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1027/2052] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:57:03.225801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.227345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:03.240968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c01f9e-8494-4b38-8105-9bc783eb7c32 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.247350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.248015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.248460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.248766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] PATH_INFO: `/v3/projects/7ffcdce933124bacba7b5b1f07b8f05d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.250165 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ffcdce933124bacba7b5b1f07b8f05d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.252121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.323279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ffcdce933124bacba7b5b1f07b8f05d', 'request_id': 'req-fe779c06-9640-452e-a8d1-4cdea5437def'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:03.335683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ffcdce933124bacba7b5b1f07b8f05d', 'request_id': 'req-fe779c06-9640-452e-a8d1-4cdea5437def'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:03.342919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ffcdce933124bacba7b5b1f07b8f05d', 'request_id': 'req-fe779c06-9640-452e-a8d1-4cdea5437def'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:03.354058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] The token cache is being invalidate because project 7ffcdce933124bacba7b5b1f07b8f05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:03.354342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:03.354990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fe779c06-9640-452e-a8d1-4cdea5437def admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:03.356523 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1026/2053] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:03 2022] DELETE /identity/v3/projects/7ffcdce933124bacba7b5b1f07b8f05d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:03.361462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.361720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.361984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.381943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:03.383307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:03.393370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:03.446282 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1028/2054] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:03.454402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.454752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.455086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.475813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:03.477080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:03.485787 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1027/2055] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:03.490463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.501303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3f9b2-56c7-495a-a8d6-816fb7cfb564 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.506126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.506745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.507021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.507319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.508645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:03.508978 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.510613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.523334 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1029/2056] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:03.527325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.536256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57a2cf-a3fb-4efc-b07f-68863e54c776 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.541080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.541689 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.541960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.542228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.542996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.545293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.578063 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1028/2057] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:03 2022] POST /identity/v3/projects => generated 391 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:03.582943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.595376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9610b-5a93-4ce8-9043-05be3144920c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.652651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.653256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.653531 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.653838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.654600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.656976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.686156 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1030/2058] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:03 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:03.688041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.698798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dec4a-cd19-4e62-b350-b70915933771 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.704338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.704943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.705214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.705467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.706096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.707805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.718214 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1029/2059] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:03.722275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.735444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029026b5-4541-4b9e-89bf-22b734f24dda admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.740368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903904c-2276-4a26-8eee-212b506f652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.740967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903904c-2276-4a26-8eee-212b506f652c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.741310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903904c-2276-4a26-8eee-212b506f652c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.741710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903904c-2276-4a26-8eee-212b506f652c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.742288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903904c-2276-4a26-8eee-212b506f652c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.743949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903904c-2276-4a26-8eee-212b506f652c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.755004 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1031/2060] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:57:03.759449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.769504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f8c3e-50c0-4181-b594-83e3fe6597ba admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:03.774590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:03.775267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.775551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.775816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] PATH_INFO: `/v3/projects/8e6c7d29b67a4965b29957a1fc3fc5be/users/553a0d3d4246496a8f91131f7e85409e/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.791573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6c7d29b67a4965b29957a1fc3fc5be, user_id=553a0d3d4246496a8f91131f7e85409e, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:03.794881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:03.813431 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1030/2061] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:03 2022] PUT /identity/v3/projects/8e6c7d29b67a4965b29957a1fc3fc5be/users/553a0d3d4246496a8f91131f7e85409e/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:03.820509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:03.820794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:03.821045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:03.852746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] MFA Rules not processed for user `553a0d3d4246496a8f91131f7e85409e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:03.855379 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:03.929830 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1032/2062] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:03.949273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:03.951663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:04.052129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db56459-6f22-4968-9b28-9000f1ea1fee admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:04.056474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd88cd-2e1b-4c96-8bbf-2c5da08dd617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:04.057283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd88cd-2e1b-4c96-8bbf-2c5da08dd617 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:04.057700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd88cd-2e1b-4c96-8bbf-2c5da08dd617 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:04.058108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd88cd-2e1b-4c96-8bbf-2c5da08dd617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:04.068292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd88cd-2e1b-4c96-8bbf-2c5da08dd617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:04.070041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd88cd-2e1b-4c96-8bbf-2c5da08dd617 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:04.084710 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1031/2063] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:03 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:07.636734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:07.638484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:07.653464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:07.719676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba7aa9-e6e9-4ef9-a34b-fed0f5910b31 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:07.724990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:07.725630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:07.725904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:07.726154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] PATH_INFO: `/v3/users/6d1ab82fe4fe41e391b27a4847449de4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:07.727411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d1ab82fe4fe41e391b27a4847449de4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:07.729392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:07.780327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d1ab82fe4fe41e391b27a4847449de4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:07.789038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d1ab82fe4fe41e391b27a4847449de4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:07.802679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d1ab82fe4fe41e391b27a4847449de4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:07.803378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d1ab82fe4fe41e391b27a4847449de4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:07.813640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d1ab82fe4fe41e391b27a4847449de4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:07.819644 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1033/2064] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:07 2022] DELETE /identity/v3/users/6d1ab82fe4fe41e391b27a4847449de4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:07.824622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:07.825134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:07.825285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:07.838650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:07.844837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:07.846516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:07.905215 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1032/2065] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:07.915396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:07.916880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:07.976390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8bacde-256d-4d33-b324-06850a525e36 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:07.980182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fbdc6-601d-4149-b2fc-33815af508ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:07.980828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fbdc6-601d-4149-b2fc-33815af508ea service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:07.981103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fbdc6-601d-4149-b2fc-33815af508ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:07.981457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fbdc6-601d-4149-b2fc-33815af508ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:07.991619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fbdc6-601d-4149-b2fc-33815af508ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:07.993356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394fbdc6-601d-4149-b2fc-33815af508ea service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.003837 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1034/2066] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:08.365433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.367215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:08.401761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d52c6d-b512-4a5b-9e10-eab08c65ac1a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.407100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.408041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.410712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.410978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] PATH_INFO: `/v3/projects/b72e36a56afe46bc94dde799bae91a89` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.414358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72e36a56afe46bc94dde799bae91a89)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.418494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.473195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72e36a56afe46bc94dde799bae91a89', 'request_id': 'req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:08.481814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72e36a56afe46bc94dde799bae91a89', 'request_id': 'req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:08.489318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72e36a56afe46bc94dde799bae91a89', 'request_id': 'req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:08.506128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] The token cache is being invalidate because project b72e36a56afe46bc94dde799bae91a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:08.506410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:08.507052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-48d6ccf5-9f25-4019-9e9e-d918fc8f4b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:08.508762 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1033/2067] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:08 2022] DELETE /identity/v3/projects/b72e36a56afe46bc94dde799bae91a89 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:08.513840 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.514119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.514421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.534139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:08.535377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:08.566594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:08.599489 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1035/2068] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:08.609210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.609453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.609722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.630167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:08.631462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:08.639925 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1034/2069] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:08.644394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.654661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490c7f-cd70-4ad1-803c-c83ddd78c63d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.660570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.661198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.661500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.661801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.662480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:08.662762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.664353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.677157 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1036/2070] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:08.681620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.691800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec1b18-7f95-4570-b9c3-d44c3a378835 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.697335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.697942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.698213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.698483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.699099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.701093 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.730738 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1035/2071] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:08 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:08.735324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.748192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efadf61-a7a2-4720-924f-1aa5132a746f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.808851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.809464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.809735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.810000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.810629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.812598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.837641 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1037/2072] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:08 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:08.838784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.852746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a420337-dc28-4264-81b7-afee4cc91a09 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.858063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.859085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.859480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.859691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.860505 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.861975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.873169 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1036/2073] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:08.881427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.891738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a00f5-6edf-47a6-b48d-06e3460fa70f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.896792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.897400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.897685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.897932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.898603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.900246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.918539 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1038/2074] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:08.920686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.931813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d162bd2-2011-47ed-80a5-aa8b66472356 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.936956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d4c46-8818-43cd-a1b5-9ae08abacd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:08.937577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d4c46-8818-43cd-a1b5-9ae08abacd95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.937932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d4c46-8818-43cd-a1b5-9ae08abacd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.938259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d4c46-8818-43cd-a1b5-9ae08abacd95 admin admin] PATH_INFO: `/v3/projects/f4512d21d91a4e1aad832b843016ad88/users/4682f54465784ce582c1002ac4f281f8/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:08.955094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:08.957162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:08.958377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d4c46-8818-43cd-a1b5-9ae08abacd95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4512d21d91a4e1aad832b843016ad88, user_id=4682f54465784ce582c1002ac4f281f8, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:08.961144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d4c46-8818-43cd-a1b5-9ae08abacd95 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:08.972438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f3a43-8632-41b5-94af-809e84c218c6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:08.976342 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1037/2075] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:08 2022] PUT /identity/v3/projects/f4512d21d91a4e1aad832b843016ad88/users/4682f54465784ce582c1002ac4f281f8/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:57:08.987692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:08.988029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:08.988308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.022127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] MFA Rules not processed for user `4682f54465784ce582c1002ac4f281f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:09.023574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.042308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:09.049784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.050448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.050845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.051018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] PATH_INFO: `/v3/users/858d7323da1b4b8a9c9c58479f75eed9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.054591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=858d7323da1b4b8a9c9c58479f75eed9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.056803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.111867 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1038/2076] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:09.117421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.121308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:09.127813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '858d7323da1b4b8a9c9c58479f75eed9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:09.136017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4cac4f-c4c2-4319-8d3c-31660050d0b8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.137782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '858d7323da1b4b8a9c9c58479f75eed9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:09.142499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.143435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.143845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.144247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.145294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.147849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.152195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '858d7323da1b4b8a9c9c58479f75eed9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:09.153062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '858d7323da1b4b8a9c9c58479f75eed9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:09.160854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3ee5bba2-04a3-4156-92f2-78d0eb300600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '858d7323da1b4b8a9c9c58479f75eed9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:09.167313 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1039/2077] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:08 2022] DELETE /identity/v3/users/858d7323da1b4b8a9c9c58479f75eed9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:09.172945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.173325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.173661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.177889 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1039/2078] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:09 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:57:09.181863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.183563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.194629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:09.195837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.249441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f11b4e-7362-44f9-be53-95847f665a58 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.254326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248543e4-d442-4993-b160-59453981435b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.254941 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248543e4-d442-4993-b160-59453981435b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.255236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248543e4-d442-4993-b160-59453981435b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.255576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248543e4-d442-4993-b160-59453981435b admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.256216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248543e4-d442-4993-b160-59453981435b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.258110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248543e4-d442-4993-b160-59453981435b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.259415 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1040/2079] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:57:09.273193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.274598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.289780 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1040/2080] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:09 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:57:09.290894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248543e4-d442-4993-b160-59453981435b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.301814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248543e4-d442-4993-b160-59453981435b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.306921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.307324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.307602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.307870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.308648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.310234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.325107 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1041/2081] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:09.327035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.339008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64029304-9818-4b99-b2d6-564f28f19cea admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.344128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.344850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4c9d2-0f6a-4a06-bdb9-232f5a64c92d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.347233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.347561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.347838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] PATH_INFO: `/v3/projects/3f623afb366147efae2e5fe429f3fee7/users/b5edebdcdfb642b9a313e2e980212e62/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.348938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.349633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.349904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.350175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.364239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.365440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f623afb366147efae2e5fe429f3fee7, user_id=b5edebdcdfb642b9a313e2e980212e62, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.365766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.368134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.377519 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1041/2082] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:57:09.383968 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1042/2083] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:09 2022] PUT /identity/v3/projects/3f623afb366147efae2e5fe429f3fee7/users/b5edebdcdfb642b9a313e2e980212e62/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:09.387767 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.399370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd47850c-b7bf-4bfa-8e0d-b3f1c14b3ae3 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.478513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba14b0b-ac4e-4ab0-8c40-cb74d89c10b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.479510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba14b0b-ac4e-4ab0-8c40-cb74d89c10b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.479948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba14b0b-ac4e-4ab0-8c40-cb74d89c10b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.480274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba14b0b-ac4e-4ab0-8c40-cb74d89c10b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.481282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba14b0b-ac4e-4ab0-8c40-cb74d89c10b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.483262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba14b0b-ac4e-4ab0-8c40-cb74d89c10b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.486247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:09.499507 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1042/2084] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:09.504316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.523240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82e2fa-e617-4a3b-a43a-a78bd8eebf0f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.526342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.527023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.527326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.527725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] PATH_INFO: `/v3/projects/3f623afb366147efae2e5fe429f3fee7/users/b5edebdcdfb642b9a313e2e980212e62/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.534867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f623afb366147efae2e5fe429f3fee7, user_id=b5edebdcdfb642b9a313e2e980212e62, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.543091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.583242 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1043/2085] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:09 2022] PUT /identity/v3/projects/3f623afb366147efae2e5fe429f3fee7/users/b5edebdcdfb642b9a313e2e980212e62/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:09.590464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.590748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.591015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.623806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] MFA Rules not processed for user `b5edebdcdfb642b9a313e2e980212e62`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:09.625008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.708255 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1043/2086] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:09.720778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.722729 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.766861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:09.786551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311e0e0-7fda-45c4-8100-86d78d8602a6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.790382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.791061 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.791522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.791793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.793940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.874767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.876167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e26a44-9d2f-41f3-8b84-a5e8616d94cd service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:09.892990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:09.892990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:09.899520 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1044/2087] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:57:09.972673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce511d-a344-498f-a06d-50690b735a7a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:09.977502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:09.978205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:09.978481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:09.978750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] PATH_INFO: `/v3/projects/02a7e1461ef24da8a1cd7d5cbb50cbae` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:09.979888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a7e1461ef24da8a1cd7d5cbb50cbae)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:09.981822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.037880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a7e1461ef24da8a1cd7d5cbb50cbae', 'request_id': 'req-95ea61a6-8134-4132-98e0-bd7627841320'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:10.050594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a7e1461ef24da8a1cd7d5cbb50cbae', 'request_id': 'req-95ea61a6-8134-4132-98e0-bd7627841320'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:10.062341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a7e1461ef24da8a1cd7d5cbb50cbae', 'request_id': 'req-95ea61a6-8134-4132-98e0-bd7627841320'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:10.076760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] The token cache is being invalidate because project 02a7e1461ef24da8a1cd7d5cbb50cbae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:10.077742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:10.078309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ea61a6-8134-4132-98e0-bd7627841320 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:10.080632 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1044/2088] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:09 2022] DELETE /identity/v3/projects/02a7e1461ef24da8a1cd7d5cbb50cbae => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:10.085959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.086343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.086742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.111120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:10.112338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:10.194330 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1045/2089] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:10.205642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.205967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.206243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.232229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:10.233730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:10.247736 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1045/2090] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:10.249533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.265870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70082986-4ce3-4012-8faf-817aa54d13aa None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.278975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.280354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.280558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.280948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.281624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:10.281910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.283624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.298503 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1046/2091] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:10.302975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.313510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea109918-4c94-4737-9945-16564fbdb9f3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.318474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.319081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.319390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.319653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.320286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.322359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.327498 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:10.355580 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1046/2092] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:10 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:10.359859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.371452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfbff6-b78b-4228-9ea4-69c435f94535 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.435148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.435454 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.435727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.435993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.436634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.438547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.465684 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1047/2093] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:10 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:10.469087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.483595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ee32-600e-41a6-82f9-9ae0e5c27378 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.491790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.492416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.492678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.492947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.493606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.495224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.506438 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1047/2094] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:10.511362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.523899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c4cfa-2c08-47a9-b865-3bf8a3307df6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.528817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacbba4c-0d93-401b-9ee4-4fc9d6c62cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.529434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbba4c-0d93-401b-9ee4-4fc9d6c62cea admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.529708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbba4c-0d93-401b-9ee4-4fc9d6c62cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.529977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacbba4c-0d93-401b-9ee4-4fc9d6c62cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.530631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbba4c-0d93-401b-9ee4-4fc9d6c62cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.532578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacbba4c-0d93-401b-9ee4-4fc9d6c62cea admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.534465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:10.547388 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1048/2095] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:10.552271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.563088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2f7803-08e5-4bb0-adc5-0d7070c1d12c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.568684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.569422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.569668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.569898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] PATH_INFO: `/v3/projects/34e8868077dc4e70b1ecbd8c9dc7b50b/users/fbcfc838ad004937b3acceac35162d25/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.585526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e8868077dc4e70b1ecbd8c9dc7b50b, user_id=fbcfc838ad004937b3acceac35162d25, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.588214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.602459 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1048/2096] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:10 2022] PUT /identity/v3/projects/34e8868077dc4e70b1ecbd8c9dc7b50b/users/fbcfc838ad004937b3acceac35162d25/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:10.612915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.613199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.613469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.641791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] MFA Rules not processed for user `fbcfc838ad004937b3acceac35162d25`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:10.642924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:10.713358 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1049/2097] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:10.725352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:10.727201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:10.787837 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906611-a1b6-41c9-ac5b-b0df31f3650a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:10.792884 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:10.793992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:10.794264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:10.794752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:10.804819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:10.806268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:10.816217 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1049/2098] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:12.560522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:12.570653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60566b-eb9f-4f1e-bfec-b78d5802f4b5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:12.576065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:12.578169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:12.578455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:12.578727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:12.580692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:12.661815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:12.664432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f8882-5985-44fe-a365-d772314e459b service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:12.687255 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1050/2099] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:14.528726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:14.530369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:14.600514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab1312-ba89-44ff-8049-c9a9e88fb55f service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:14.605549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:14.606194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:14.606500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:14.606777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] PATH_INFO: `/v3/users/553a0d3d4246496a8f91131f7e85409e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:14.607932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=553a0d3d4246496a8f91131f7e85409e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:14.610039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:14.663626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '553a0d3d4246496a8f91131f7e85409e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:14.672537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '553a0d3d4246496a8f91131f7e85409e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:14.673749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:14.687474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '553a0d3d4246496a8f91131f7e85409e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:14.688102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '553a0d3d4246496a8f91131f7e85409e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:14.694705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '553a0d3d4246496a8f91131f7e85409e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:14.700420 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1050/2100] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:14 2022] DELETE /identity/v3/users/553a0d3d4246496a8f91131f7e85409e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:14.705135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:14.705598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:14.705707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:14.725715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:14.726925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:14.791770 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1051/2101] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:14.802043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:14.803545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:14.863306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365bfd-4c2a-4a20-b3d5-ac820bc37916 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:14.868225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf66bca-3ecf-490f-b552-61904c5c31aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:14.868838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66bca-3ecf-490f-b552-61904c5c31aa service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:14.869107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66bca-3ecf-490f-b552-61904c5c31aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:14.869367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66bca-3ecf-490f-b552-61904c5c31aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:14.879271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf66bca-3ecf-490f-b552-61904c5c31aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:14.880700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf66bca-3ecf-490f-b552-61904c5c31aa service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:14.890672 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1051/2102] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:15.167116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:15.169267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:15.182085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e1f2d-7b3c-40f8-b580-2f24bb349bb9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:15.188076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:15.189235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:15.189235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:15.189413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] PATH_INFO: `/v3/projects/8e6c7d29b67a4965b29957a1fc3fc5be` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:15.190611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6c7d29b67a4965b29957a1fc3fc5be)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:15.192635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:15.209809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:15.245073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6c7d29b67a4965b29957a1fc3fc5be', 'request_id': 'req-0b9443c6-341e-407a-b8af-101e3d02b15e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:15.251983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6c7d29b67a4965b29957a1fc3fc5be', 'request_id': 'req-0b9443c6-341e-407a-b8af-101e3d02b15e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:15.259330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6c7d29b67a4965b29957a1fc3fc5be', 'request_id': 'req-0b9443c6-341e-407a-b8af-101e3d02b15e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:15.276895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] The token cache is being invalidate because project 8e6c7d29b67a4965b29957a1fc3fc5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:15.277178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:15.278749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:15.280716 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1052/2103] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:15 2022] DELETE /identity/v3/projects/8e6c7d29b67a4965b29957a1fc3fc5be => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:18.109017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69992-10e2-467b-a1d6-47891169c0f0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:18.109454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69992-10e2-467b-a1d6-47891169c0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:18.109541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b69992-10e2-467b-a1d6-47891169c0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:18.130344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d2b69992-10e2-467b-a1d6-47891169c0f0 None None] MFA Rules not processed for user `398da9e11a194b9f9644dfad517d07b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:18.131629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d2b69992-10e2-467b-a1d6-47891169c0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:18.193166 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1052/2104] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:25.270159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:25.635337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:25.636862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:25.713695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9443c6-341e-407a-b8af-101e3d02b15e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:25.719451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:25.720058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:25.720332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:25.720597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] PATH_INFO: `/v3/users/4682f54465784ce582c1002ac4f281f8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:25.721830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4682f54465784ce582c1002ac4f281f8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:25.723809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:25.778453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4682f54465784ce582c1002ac4f281f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:25.786203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4682f54465784ce582c1002ac4f281f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:25.794031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4682f54465784ce582c1002ac4f281f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:25.795950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4682f54465784ce582c1002ac4f281f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:25.810102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4682f54465784ce582c1002ac4f281f8'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:25.816666 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1053/2105] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:25 2022] DELETE /identity/v3/users/4682f54465784ce582c1002ac4f281f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:25.821253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:25.821742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:25.821852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:25.841766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:25.842975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:25.903000 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1053/2106] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:25.914727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:25.916925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:25.975216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4ffbf-b13f-4e83-b531-f8144b71324d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:25.979057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:25.979697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:25.979966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:25.980231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:25.990424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:25.991591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:26.002106 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1054/2107] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:26.402220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:26.403802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:26.419860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee32728-9831-4031-98c5-36483000ae76 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:26.425295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:26.425936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:26.426207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:26.426471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] PATH_INFO: `/v3/projects/f4512d21d91a4e1aad832b843016ad88` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:26.427651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4512d21d91a4e1aad832b843016ad88)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:26.430068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:26.458110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:26.495888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4512d21d91a4e1aad832b843016ad88', 'request_id': 'req-536e9c6f-8743-4211-8164-0fe2045f41c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.506062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4512d21d91a4e1aad832b843016ad88', 'request_id': 'req-536e9c6f-8743-4211-8164-0fe2045f41c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.521657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4512d21d91a4e1aad832b843016ad88', 'request_id': 'req-536e9c6f-8743-4211-8164-0fe2045f41c4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.535573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] The token cache is being invalidate because project f4512d21d91a4e1aad832b843016ad88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:26.535887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.536560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.538856 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1054/2108] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:26 2022] DELETE /identity/v3/projects/f4512d21d91a4e1aad832b843016ad88 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:26.543206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:26.559385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022aae68-8509-4f99-aebc-6cec4e7269d8 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:26.589929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:26.633128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:26.633789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:26.634137 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:26.634413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] PATH_INFO: `/v3/users/b5edebdcdfb642b9a313e2e980212e62` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:26.635704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5edebdcdfb642b9a313e2e980212e62)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:26.637660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:26.688988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5edebdcdfb642b9a313e2e980212e62'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.697920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5edebdcdfb642b9a313e2e980212e62'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.705119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5edebdcdfb642b9a313e2e980212e62'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.705747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5edebdcdfb642b9a313e2e980212e62'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.717251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ee976a05-8ac1-467d-986d-52a663c91e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5edebdcdfb642b9a313e2e980212e62'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:26.723360 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1055/2109] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:26 2022] DELETE /identity/v3/users/b5edebdcdfb642b9a313e2e980212e62 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:27.329439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.331170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:27.359626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e9c6f-8743-4211-8164-0fe2045f41c4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.426229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.426889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.427158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.427446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] PATH_INFO: `/v3/projects/3f623afb366147efae2e5fe429f3fee7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.428627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f623afb366147efae2e5fe429f3fee7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.430566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.481616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f623afb366147efae2e5fe429f3fee7', 'request_id': 'req-2a07751c-fcf2-4ecf-9433-9df851b02396'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:27.490255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f623afb366147efae2e5fe429f3fee7', 'request_id': 'req-2a07751c-fcf2-4ecf-9433-9df851b02396'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:27.496016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f623afb366147efae2e5fe429f3fee7', 'request_id': 'req-2a07751c-fcf2-4ecf-9433-9df851b02396'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:27.507985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] The token cache is being invalidate because project 3f623afb366147efae2e5fe429f3fee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:27.508193 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:27.508827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2a07751c-fcf2-4ecf-9433-9df851b02396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:27.510321 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1055/2110] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:27 2022] DELETE /identity/v3/projects/3f623afb366147efae2e5fe429f3fee7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:27.515450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.515735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.516033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.536871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:27.538085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:27.601118 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1056/2111] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:27.609471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.609749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.610022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.631149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:27.632390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:27.643906 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1056/2112] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:27.648325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.658599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d32d66-e94e-479f-a0fa-bd0ccf13e210 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.663810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.664410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.664713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.665035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.665707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:27.666017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.667666 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.680230 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1057/2113] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 00:57:27.685221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.694757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c04219-a7e9-40c4-9b06-0b09ff09e05b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.699917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.700526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.700798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.701063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.701693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.703797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.735622 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1057/2114] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:27 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:27.740581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.754507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c5b8b-ee73-47e6-92a1-8b528afc9c68 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.785952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:27.818796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.819428 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.819701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.819970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.820603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.822516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.850897 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1058/2115] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:27 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:27.856707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.865506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:27.870056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629282-92ce-43d1-b57b-2179fa423891 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.876758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.877407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.877886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.878368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.879047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.880855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.892480 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1058/2116] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:57:27.896384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.907156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9dbf9-bf42-4c6f-8ee5-fab25de42b85 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.913029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f900d27-676f-4708-8dd6-876bc5706127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.913630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f900d27-676f-4708-8dd6-876bc5706127 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.913905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f900d27-676f-4708-8dd6-876bc5706127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.914172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f900d27-676f-4708-8dd6-876bc5706127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.914821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f900d27-676f-4708-8dd6-876bc5706127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.916460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f900d27-676f-4708-8dd6-876bc5706127 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.927921 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1059/2117] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:27.932373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:27.944073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80718de-afe1-4024-b25c-08c5c5cca9b2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:27.949795 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:27.950450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.950719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.950975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] PATH_INFO: `/v3/projects/a07b912baaf344af876eaa6022b6050f/users/aafc5bb81f044cf3bf5dcacb1bc0cd93/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:27.966078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07b912baaf344af876eaa6022b6050f, user_id=aafc5bb81f044cf3bf5dcacb1bc0cd93, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:27.968833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:27.984899 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1059/2118] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:27 2022] PUT /identity/v3/projects/a07b912baaf344af876eaa6022b6050f/users/aafc5bb81f044cf3bf5dcacb1bc0cd93/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:27.995696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:27.995980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:27.996260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.025471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] MFA Rules not processed for user `aafc5bb81f044cf3bf5dcacb1bc0cd93`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:28.026709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:28.100202 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1060/2119] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:27 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:28.104827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.174644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccbe3e-9b8f-48b3-b2f8-dee92cdb6c00 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.179616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.180319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.180678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.181011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.181733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.183744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.213559 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1060/2120] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:28 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:28.217611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.228444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203f6eeb-f07a-4563-a20d-aff6c89247e8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.290364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.290989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.291288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.291556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.292202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.294467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.323368 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1061/2121] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:28 2022] POST /identity/v3/users => generated 302 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:28.327656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.343398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795ff8-261f-41ee-b287-8ec247a00146 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.348442 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.349052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.349330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.349593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.350238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.351879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.362682 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1061/2122] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:28.367061 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.379320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ebe02-0c15-418e-ace7-a5c0cf341d52 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.383891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.384781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.385254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.385605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] PATH_INFO: `/v3/projects/218efe2be6514df48eb820e7cdb4455a/users/691d67fa28b34210bfc5d862ee4c17f3/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.401599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] RBAC: Authorizing `identity:create_grant(project_id=218efe2be6514df48eb820e7cdb4455a, user_id=691d67fa28b34210bfc5d862ee4c17f3, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.404351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.420137 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1062/2123] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:28 2022] PUT /identity/v3/projects/218efe2be6514df48eb820e7cdb4455a/users/691d67fa28b34210bfc5d862ee4c17f3/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:28.424254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.434645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225c8db-0159-444b-8e2e-ac41ba2c87f4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.500499 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615b623f-45cc-426e-96d5-aad31509793c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.501116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b623f-45cc-426e-96d5-aad31509793c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.501389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b623f-45cc-426e-96d5-aad31509793c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.501696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615b623f-45cc-426e-96d5-aad31509793c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.502346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b623f-45cc-426e-96d5-aad31509793c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.503994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615b623f-45cc-426e-96d5-aad31509793c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.516223 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1062/2124] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:28.520643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.526205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:28.532549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1732-5ce9-417b-babe-130bf0a3075a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.538595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.539408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.540448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.540448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] PATH_INFO: `/v3/projects/218efe2be6514df48eb820e7cdb4455a/users/691d67fa28b34210bfc5d862ee4c17f3/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.547289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=218efe2be6514df48eb820e7cdb4455a, user_id=691d67fa28b34210bfc5d862ee4c17f3, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.551033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.566279 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1063/2125] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:28 2022] PUT /identity/v3/projects/218efe2be6514df48eb820e7cdb4455a/users/691d67fa28b34210bfc5d862ee4c17f3/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:28.577791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.578755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.579028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.612354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] MFA Rules not processed for user `691d67fa28b34210bfc5d862ee4c17f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:28.613640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:28.663649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:28.715488 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1063/2126] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:28.726763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.728398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:28.802266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a57cf0-cb6a-411f-a955-530110f5ab3b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.806460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.807324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.807607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.807881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.819948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.821434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.833028 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1064/2127] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:28.888530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:28.889934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:28.901575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531dd21-9d86-49cd-b813-eee81083c4b3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:28.905115 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558b3a4-f144-48a2-ba41-cfb7c6eaeaef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:28.905709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b3a4-f144-48a2-ba41-cfb7c6eaeaef service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:28.905982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b3a4-f144-48a2-ba41-cfb7c6eaeaef service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:28.906248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558b3a4-f144-48a2-ba41-cfb7c6eaeaef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:28.963910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558b3a4-f144-48a2-ba41-cfb7c6eaeaef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:28.965344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558b3a4-f144-48a2-ba41-cfb7c6eaeaef service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:28.974327 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1064/2128] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:32.462279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:32.531130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b61360e-8082-410f-90d0-844bc1abbce1 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:32.535854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:32.536508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:32.536781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:32.537048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] PATH_INFO: `/v3/users/aafc5bb81f044cf3bf5dcacb1bc0cd93` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:32.538205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aafc5bb81f044cf3bf5dcacb1bc0cd93)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:32.540596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:32.591831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aafc5bb81f044cf3bf5dcacb1bc0cd93'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:32.602101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aafc5bb81f044cf3bf5dcacb1bc0cd93'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:32.610527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aafc5bb81f044cf3bf5dcacb1bc0cd93'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:32.614032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aafc5bb81f044cf3bf5dcacb1bc0cd93'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:32.629769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aafc5bb81f044cf3bf5dcacb1bc0cd93'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:32.641485 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1065/2129] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:32 2022] DELETE /identity/v3/users/aafc5bb81f044cf3bf5dcacb1bc0cd93 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:32.643799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:32.645042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:32.645533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:32.671920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:32.673339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:32.744743 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1065/2130] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:32.754473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:32.756005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:32.769423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:32.838716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a847ed2-2ed8-4981-8a2c-85a79927cc1e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:32.842970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:32.843932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:32.843932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:32.844214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:32.854470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:32.855984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:32.866856 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1066/2131] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:33.371917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:33.373464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:33.386145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f2d53-9cc0-49c3-ae67-4ff9f399ed93 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:33.392081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:33.392830 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:33.393200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:33.393747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] PATH_INFO: `/v3/projects/a07b912baaf344af876eaa6022b6050f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:33.394756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07b912baaf344af876eaa6022b6050f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:33.397928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:33.480975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07b912baaf344af876eaa6022b6050f', 'request_id': 'req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.491485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07b912baaf344af876eaa6022b6050f', 'request_id': 'req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.495529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:33.505700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07b912baaf344af876eaa6022b6050f', 'request_id': 'req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.519819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] The token cache is being invalidate because project a07b912baaf344af876eaa6022b6050f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:33.520730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.521043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.522681 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1066/2132] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:33 2022] DELETE /identity/v3/projects/a07b912baaf344af876eaa6022b6050f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:33.527039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:33.542142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442c3bd-2e81-467d-9289-ef4bbefcb64c service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:33.622800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:33.624269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:33.624545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:33.624814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] PATH_INFO: `/v3/users/691d67fa28b34210bfc5d862ee4c17f3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:33.625992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=691d67fa28b34210bfc5d862ee4c17f3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:33.628095 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:33.683487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '691d67fa28b34210bfc5d862ee4c17f3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.700353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '691d67fa28b34210bfc5d862ee4c17f3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.708347 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '691d67fa28b34210bfc5d862ee4c17f3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.709285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '691d67fa28b34210bfc5d862ee4c17f3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.722609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-69977f2e-d60c-4b48-a204-0bb525e788bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '691d67fa28b34210bfc5d862ee4c17f3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:33.733811 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1067/2133] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:33 2022] DELETE /identity/v3/users/691d67fa28b34210bfc5d862ee4c17f3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:34.177788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.179530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:34.198346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d9389-6fe1-4c6a-9c9a-8f0c8d30e662 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.278157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.278719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.278986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.279280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] PATH_INFO: `/v3/projects/218efe2be6514df48eb820e7cdb4455a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.280765 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=218efe2be6514df48eb820e7cdb4455a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.282789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.346120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218efe2be6514df48eb820e7cdb4455a', 'request_id': 'req-285a855e-ed42-4b82-9e74-6052f70cb526'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:34.355762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218efe2be6514df48eb820e7cdb4455a', 'request_id': 'req-285a855e-ed42-4b82-9e74-6052f70cb526'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:34.362183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218efe2be6514df48eb820e7cdb4455a', 'request_id': 'req-285a855e-ed42-4b82-9e74-6052f70cb526'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:34.374090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] The token cache is being invalidate because project 218efe2be6514df48eb820e7cdb4455a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:34.374370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:34.375011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-285a855e-ed42-4b82-9e74-6052f70cb526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:34.376512 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1067/2134] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:34 2022] DELETE /identity/v3/projects/218efe2be6514df48eb820e7cdb4455a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:34.382053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.382325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.382590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.402259 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:34.403492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:34.433774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:34.474281 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1068/2135] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:34.482243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.482519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.482814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.508570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:34.509916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:34.521282 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1068/2136] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:34.526060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.539701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632852ee-9dc9-4500-ad33-7498771c1064 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.546261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.546890 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.547170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.547456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.548136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:34.548436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.550037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.564797 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1069/2137] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:34.569746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.580691 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7becfe1-764f-4a1d-93da-6ae704925fca None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.585733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.586390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.590577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.590577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.590577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.590577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.617885 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1069/2138] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:34 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:34.622513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.638595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a0ec8-fdad-493b-8160-f853954cf225 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.697434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.698038 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.698310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.698578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.699249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.701189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.724636 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1070/2139] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:34 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:57:34.729060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.739805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cd75e-a3df-4eb5-98f2-d9dc05a06ed1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.745816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.746416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.746685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.747428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.748423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.750505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.762510 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1070/2140] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:34.766726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.777648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5e4aa-609c-4c71-858c-2f0013cbfc64 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.783434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8769b-69c1-4d69-b256-64c4e891a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.784181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8769b-69c1-4d69-b256-64c4e891a1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.784485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8769b-69c1-4d69-b256-64c4e891a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.784758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8769b-69c1-4d69-b256-64c4e891a1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.785434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8769b-69c1-4d69-b256-64c4e891a1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.787155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8769b-69c1-4d69-b256-64c4e891a1b6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.802214 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1071/2141] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:34.803713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:34.814956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0bdf6d-90c4-4b0b-a15a-ae4450c00efa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:34.819635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:34.820295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.820579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.820870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] PATH_INFO: `/v3/projects/4eec00ad3dac435b947ab46328af0235/users/6ac8b73f4f05419686e5b860eb718b76/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.839082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eec00ad3dac435b947ab46328af0235, user_id=6ac8b73f4f05419686e5b860eb718b76, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:34.842878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:34.877662 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1071/2142] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:34 2022] PUT /identity/v3/projects/4eec00ad3dac435b947ab46328af0235/users/6ac8b73f4f05419686e5b860eb718b76/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:34.879819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:34.880104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:34.880381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:34.919195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] MFA Rules not processed for user `6ac8b73f4f05419686e5b860eb718b76`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:34.920485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:35.020888 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1072/2143] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:35.022447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.052784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:35.109296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b46ac8-8eb5-4663-9a64-e2b3a1a98752 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.114420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.115056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.115361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.115669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.116316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.118402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.151556 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1072/2144] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:35 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:35.155858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.170192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca4e462-8dba-4215-a56d-a6d79d42863d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.207813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:35.242673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.243164 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.243471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.243785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.244453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.246335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.268005 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1073/2145] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:35 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:35.278264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.289542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89196b28-b031-468d-828d-86e32de16e96 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.295292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.295980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.296387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.296661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.297303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.298926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.310224 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1073/2146] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:35.314130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.324202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525b131-deb5-48a4-934d-23ef6cb5eed5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.328826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a8e96-efd2-480e-883f-87a5a1be7a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.329433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a8e96-efd2-480e-883f-87a5a1be7a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.329706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a8e96-efd2-480e-883f-87a5a1be7a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.329971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a8e96-efd2-480e-883f-87a5a1be7a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.330620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a8e96-efd2-480e-883f-87a5a1be7a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.332253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a8e96-efd2-480e-883f-87a5a1be7a97 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.342778 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1074/2147] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:35.347671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.358410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d5ccf-dff5-4727-b151-42d6023973aa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.364591 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.365380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.365684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.365946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] PATH_INFO: `/v3/projects/d5829c66d11d4cc68512c4fe07506fe6/users/7903f70bfb7d417da6556765cf21e9f4/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.383069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5829c66d11d4cc68512c4fe07506fe6, user_id=7903f70bfb7d417da6556765cf21e9f4, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.385838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.401761 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1074/2148] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:35 2022] PUT /identity/v3/projects/d5829c66d11d4cc68512c4fe07506fe6/users/7903f70bfb7d417da6556765cf21e9f4/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:35.413809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.414087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.414408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.444637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] MFA Rules not processed for user `7903f70bfb7d417da6556765cf21e9f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:35.445945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:35.515237 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1075/2149] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:35.521312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.587385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b548c1-fcbe-432b-8ca9-d34e7726108d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.594447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.595081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.595339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.595643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.596294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.598175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.628973 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1075/2150] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:35 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:35.633439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.652789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1b6a0-ff4b-40b0-858f-872823cdf802 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.730623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.731277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.731609 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.731900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.732614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.734695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.760776 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1076/2151] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:35 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:35.765060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.776456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2219f9-15e5-425f-af6a-6830196b931e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.781532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.782146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.782483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.782740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.783540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.785359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.800990 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1076/2152] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:57:35.802287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.815357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171933a6-3362-4366-b756-5ff0414cff16 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.820311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.821015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.821290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.821560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] PATH_INFO: `/v3/projects/ba22ffa6de3c4f9da668815ea0eed3dd/users/83f75124a10b4d93b2fcbfda23ad294b/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.838360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba22ffa6de3c4f9da668815ea0eed3dd, user_id=83f75124a10b4d93b2fcbfda23ad294b, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.841235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.857654 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1077/2153] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:35 2022] PUT /identity/v3/projects/ba22ffa6de3c4f9da668815ea0eed3dd/users/83f75124a10b4d93b2fcbfda23ad294b/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:35.864756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.883841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea224a-41d5-48f8-80aa-a16b3c1a44ad admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.926140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:35.953194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ec0e-761c-4dbd-ba20-39540447aae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.953702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ec0e-761c-4dbd-ba20-39540447aae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.953975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ec0e-761c-4dbd-ba20-39540447aae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.954238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ec0e-761c-4dbd-ba20-39540447aae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.954879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ec0e-761c-4dbd-ba20-39540447aae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:35.956551 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ec0e-761c-4dbd-ba20-39540447aae3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:35.969920 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1077/2154] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:35.970969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:35.976619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:35.987069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f29254-2a25-4db7-9741-43aa260c3944 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:35.994179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:35.994869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:35.995285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:35.995523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] PATH_INFO: `/v3/projects/ba22ffa6de3c4f9da668815ea0eed3dd/users/83f75124a10b4d93b2fcbfda23ad294b/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:35.998119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba22ffa6de3c4f9da668815ea0eed3dd, user_id=83f75124a10b4d93b2fcbfda23ad294b, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:36.001012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:36.022222 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1078/2155] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:35 2022] PUT /identity/v3/projects/ba22ffa6de3c4f9da668815ea0eed3dd/users/83f75124a10b4d93b2fcbfda23ad294b/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:36.038496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:36.038902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:36.039197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:36.084065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] MFA Rules not processed for user `83f75124a10b4d93b2fcbfda23ad294b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:36.085827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:36.190044 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1078/2156] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:57:36.203097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:36.204920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:36.279897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c7d243-e9a5-4518-ab94-e64e3ff202c8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:36.283929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25d91d-ca8a-496a-a191-46f029646828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:36.284552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25d91d-ca8a-496a-a191-46f029646828 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:36.284825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25d91d-ca8a-496a-a191-46f029646828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:36.285094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25d91d-ca8a-496a-a191-46f029646828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:36.299288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25d91d-ca8a-496a-a191-46f029646828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:36.300754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25d91d-ca8a-496a-a191-46f029646828 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:36.310464 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1079/2157] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:36.372056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:36.383817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216bb75-ee9c-48c0-bac0-eeb58b32217c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:36.387707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:36.388971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:36.388971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:36.388971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:36.467493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:36.469401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:36.480914 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1079/2158] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:37.624195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b896041-dc5d-45b9-8bdc-5dace6e54c16 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:37.624715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b896041-dc5d-45b9-8bdc-5dace6e54c16 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:37.624894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b896041-dc5d-45b9-8bdc-5dace6e54c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:37.651879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-2b896041-dc5d-45b9-8bdc-5dace6e54c16 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:37.653542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2b896041-dc5d-45b9-8bdc-5dace6e54c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:37.728948 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1080/2159] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:57:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:37.777658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:37.779997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:37.843565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:37.860144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6d35f-7237-4027-ac9b-325ae1c73aa6 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:37.865455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:37.866175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:37.866446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:37.866700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:37.875679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:37.877371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:37.885908 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1080/2160] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:57:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:57:38.105736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9549e8-b8b2-4632-89f2-53e1de20a27d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:38.106330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9549e8-b8b2-4632-89f2-53e1de20a27d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:38.107402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9549e8-b8b2-4632-89f2-53e1de20a27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:38.141399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6f9549e8-b8b2-4632-89f2-53e1de20a27d None None] MFA Rules not processed for user `364cf4a7a3bf4e70973c78a5d690e998`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:38.144622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f9549e8-b8b2-4632-89f2-53e1de20a27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:38.251519 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1081/2161] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:39.621271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:39.698260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b349262-5b88-447e-a65c-cefd5fd56c78 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:39.704165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:39.705078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:39.705355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:39.705749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] PATH_INFO: `/v3/users/6ac8b73f4f05419686e5b860eb718b76` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:39.707275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ac8b73f4f05419686e5b860eb718b76)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:39.709403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:39.772170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ac8b73f4f05419686e5b860eb718b76'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:39.784622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ac8b73f4f05419686e5b860eb718b76'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:39.800365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ac8b73f4f05419686e5b860eb718b76'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:39.801334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ac8b73f4f05419686e5b860eb718b76'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:39.810895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ac8b73f4f05419686e5b860eb718b76'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:39.820756 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1081/2162] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:39 2022] DELETE /identity/v3/users/6ac8b73f4f05419686e5b860eb718b76 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:39.825502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:39.826029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:39.826029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:39.846572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:39.847830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:39.938383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:39.969465 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1082/2163] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:57:39.984302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:39.988933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:40.081924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95f954-e887-4907-98cf-8d923def9d3d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:40.087617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:40.088753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:40.089104 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:40.089522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:40.100006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:40.102080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:40.115481 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1082/2164] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:40.398802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:40.400803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:40.415161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe3c7f-f19f-4e41-af02-ce5477115124 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:40.421238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:40.421906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:40.422193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:40.422447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] PATH_INFO: `/v3/projects/4eec00ad3dac435b947ab46328af0235` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:40.423701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eec00ad3dac435b947ab46328af0235)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:40.425789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:40.483030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eec00ad3dac435b947ab46328af0235', 'request_id': 'req-3e92a535-d099-4e2f-a18f-d13a72c7501e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.493603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eec00ad3dac435b947ab46328af0235', 'request_id': 'req-3e92a535-d099-4e2f-a18f-d13a72c7501e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.512875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eec00ad3dac435b947ab46328af0235', 'request_id': 'req-3e92a535-d099-4e2f-a18f-d13a72c7501e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.527243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] The token cache is being invalidate because project 4eec00ad3dac435b947ab46328af0235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:40.527650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.528410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.530264 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1083/2165] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:40 2022] DELETE /identity/v3/projects/4eec00ad3dac435b947ab46328af0235 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:40.535773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:40.547315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca82a54-4454-455c-8015-48fc0e2d0398 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:40.554552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:40.622155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:40.622894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:40.623060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:40.623396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] PATH_INFO: `/v3/users/7903f70bfb7d417da6556765cf21e9f4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:40.624563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7903f70bfb7d417da6556765cf21e9f4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:40.626485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:40.683907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7903f70bfb7d417da6556765cf21e9f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.693541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7903f70bfb7d417da6556765cf21e9f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.707664 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7903f70bfb7d417da6556765cf21e9f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.708338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7903f70bfb7d417da6556765cf21e9f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.716218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7903f70bfb7d417da6556765cf21e9f4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:40.724344 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1083/2166] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:40 2022] DELETE /identity/v3/users/7903f70bfb7d417da6556765cf21e9f4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:41.073839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:41.075630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:41.093807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a535-d099-4e2f-a18f-d13a72c7501e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:41.198343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:41.199502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:41.199837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:41.200389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] PATH_INFO: `/v3/projects/d5829c66d11d4cc68512c4fe07506fe6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:41.202148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5829c66d11d4cc68512c4fe07506fe6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:41.207604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:41.296166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5829c66d11d4cc68512c4fe07506fe6', 'request_id': 'req-fcd05161-25e7-4de9-a300-026447aaeddc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.307671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5829c66d11d4cc68512c4fe07506fe6', 'request_id': 'req-fcd05161-25e7-4de9-a300-026447aaeddc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.315468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5829c66d11d4cc68512c4fe07506fe6', 'request_id': 'req-fcd05161-25e7-4de9-a300-026447aaeddc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.321024 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:41.330492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] The token cache is being invalidate because project d5829c66d11d4cc68512c4fe07506fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:41.330963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.332081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.333988 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1084/2167] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:41 2022] DELETE /identity/v3/projects/d5829c66d11d4cc68512c4fe07506fe6 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:41.338230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:41.340725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:41.358006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adee90d-367c-463b-894e-e9c8af346539 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:41.433323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:41.434302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:41.434781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:41.435286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] PATH_INFO: `/v3/users/83f75124a10b4d93b2fcbfda23ad294b` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:41.436813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f75124a10b4d93b2fcbfda23ad294b)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:41.440047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:41.488927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:41.508865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f75124a10b4d93b2fcbfda23ad294b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.518968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f75124a10b4d93b2fcbfda23ad294b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.535640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f75124a10b4d93b2fcbfda23ad294b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.536497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f75124a10b4d93b2fcbfda23ad294b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.545994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a81a3c72-d3a1-445f-88da-66849eb31d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f75124a10b4d93b2fcbfda23ad294b'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:41.552474 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1084/2168] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:41 2022] DELETE /identity/v3/users/83f75124a10b4d93b2fcbfda23ad294b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:42.252089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:42.256364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:42.349612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05161-25e7-4de9-a300-026447aaeddc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:42.571088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:42.571877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:42.572213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:42.572540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] PATH_INFO: `/v3/projects/ba22ffa6de3c4f9da668815ea0eed3dd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:42.573852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba22ffa6de3c4f9da668815ea0eed3dd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:42.577969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:42.676776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba22ffa6de3c4f9da668815ea0eed3dd', 'request_id': 'req-b2e7530e-a12c-4408-a81b-aef807e1d370'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:42.693913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba22ffa6de3c4f9da668815ea0eed3dd', 'request_id': 'req-b2e7530e-a12c-4408-a81b-aef807e1d370'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:42.724773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba22ffa6de3c4f9da668815ea0eed3dd', 'request_id': 'req-b2e7530e-a12c-4408-a81b-aef807e1d370'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:42.758329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] The token cache is being invalidate because project ba22ffa6de3c4f9da668815ea0eed3dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:42.758752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:42.759529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b2e7530e-a12c-4408-a81b-aef807e1d370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:42.762645 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1085/2169] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:42 2022] DELETE /identity/v3/projects/ba22ffa6de3c4f9da668815ea0eed3dd => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:42.769064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:42.769064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:42.769064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:42.809947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:42.813856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:42.943379 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1085/2170] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:42.951002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:42.951472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:42.952002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.008515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:43.009835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:43.025376 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1086/2171] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:43.026636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.048956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a185fc2-17ef-48b0-b7f8-1dd9e209a8bd None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.086951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.088931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.089436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.090067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.090901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:43.092085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.094119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.110765 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1086/2172] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:43.115733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.127022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e964ce-4d1e-4463-a558-1d53af5556d8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.146743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.148543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.148543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.148543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.149467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.155821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.191402 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1087/2173] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:43 2022] POST /identity/v3/projects => generated 383 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:43.194693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.205684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf8d6-ead6-4641-a169-539f85881523 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.396841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.397859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.398554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.399169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.400362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.402640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.448272 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1087/2174] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:43 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:43.453416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.471670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb8bc4-2387-4465-80e2-b9a711ccf5c2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.482863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.483684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.484032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.484789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.485601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.487564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.511399 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1088/2175] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:43.513954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.530923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f3e7f-84e8-4fa4-8205-81120c0ce51c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.543668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede124-5f8a-4e1e-9297-457f02b2bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.544305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede124-5f8a-4e1e-9297-457f02b2bee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.556064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede124-5f8a-4e1e-9297-457f02b2bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.556573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede124-5f8a-4e1e-9297-457f02b2bee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.557301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edede124-5f8a-4e1e-9297-457f02b2bee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.558929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edede124-5f8a-4e1e-9297-457f02b2bee0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.581828 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1088/2176] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:43.583416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.595938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c0bf8-1a25-4f2a-b481-7e7aec1fbe78 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.600953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.601625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.602029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.602621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] PATH_INFO: `/v3/projects/df7242b8c75e4970af70ff56f7d7e8bd/users/dd5b2274106a41babafec34d9ed6a635/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.620993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7242b8c75e4970af70ff56f7d7e8bd, user_id=dd5b2274106a41babafec34d9ed6a635, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.626191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.646339 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1089/2177] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:43 2022] PUT /identity/v3/projects/df7242b8c75e4970af70ff56f7d7e8bd/users/dd5b2274106a41babafec34d9ed6a635/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:43.658850 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.659382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.659756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.690283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:43.696518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] MFA Rules not processed for user `dd5b2274106a41babafec34d9ed6a635`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:43.713021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:43.810435 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1089/2178] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:43.821923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:43.828293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:43.854564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:43.891073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660be0f3-c488-4041-95c4-c52564fb2a86 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:43.895385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c4a46-7c7d-44c7-9cc1-adbcba170493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:43.896146 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4a46-7c7d-44c7-9cc1-adbcba170493 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:43.896287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4a46-7c7d-44c7-9cc1-adbcba170493 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:43.896520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c4a46-7c7d-44c7-9cc1-adbcba170493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:43.906378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c4a46-7c7d-44c7-9cc1-adbcba170493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:43.907964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c4a46-7c7d-44c7-9cc1-adbcba170493 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:43.927409 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1090/2179] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:43.940742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:44.026209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39015b0d-f214-4e20-a485-87106dadc28a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:44.032103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:44.032360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:44.032703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:44.033033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] PATH_INFO: `/v3/users/dd5b2274106a41babafec34d9ed6a635` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:44.034238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5b2274106a41babafec34d9ed6a635)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:44.036257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:44.085727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5b2274106a41babafec34d9ed6a635'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:44.092944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5b2274106a41babafec34d9ed6a635'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:44.105369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5b2274106a41babafec34d9ed6a635'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:44.106006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5b2274106a41babafec34d9ed6a635'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:44.112988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5b2274106a41babafec34d9ed6a635'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:44.118797 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1090/2180] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:43 2022] DELETE /identity/v3/users/dd5b2274106a41babafec34d9ed6a635 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:44.123339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:44.123623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:44.123893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:44.146350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:44.147620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:44.214355 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1091/2181] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:44.226532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:44.229107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:44.261256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:44.301774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833aabbe-fb0b-4647-a098-d1e7c9b703b9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:44.310852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8c70a9-2f67-4513-9801-243ec632f7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:44.311389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c70a9-2f67-4513-9801-243ec632f7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:44.311667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c70a9-2f67-4513-9801-243ec632f7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:44.311944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8c70a9-2f67-4513-9801-243ec632f7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:44.325530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c70a9-2f67-4513-9801-243ec632f7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:44.327962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8c70a9-2f67-4513-9801-243ec632f7e9 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:44.342260 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1091/2182] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:45.710856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:45.713150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:45.732917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64733b7-c779-4387-8af0-a35ee5bbad90 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:45.747068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:45.748053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:45.748563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:45.749053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] PATH_INFO: `/v3/projects/df7242b8c75e4970af70ff56f7d7e8bd` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:45.750483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7242b8c75e4970af70ff56f7d7e8bd)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:45.752875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:45.826765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7242b8c75e4970af70ff56f7d7e8bd', 'request_id': 'req-62fd84ed-b597-4aee-9576-721f629b067d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:45.845995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7242b8c75e4970af70ff56f7d7e8bd', 'request_id': 'req-62fd84ed-b597-4aee-9576-721f629b067d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:45.855295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7242b8c75e4970af70ff56f7d7e8bd', 'request_id': 'req-62fd84ed-b597-4aee-9576-721f629b067d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:45.873706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] The token cache is being invalidate because project df7242b8c75e4970af70ff56f7d7e8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:45.873998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:45.875015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-62fd84ed-b597-4aee-9576-721f629b067d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:45.882098 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1092/2183] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:45 2022] DELETE /identity/v3/projects/df7242b8c75e4970af70ff56f7d7e8bd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:45.883337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:45.883832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:45.883909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:45.922503 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:45.923781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:46.064737 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1092/2184] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:57:46.066319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.066596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.066863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.101840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:46.103013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:46.122661 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1093/2185] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:46.128748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:46.149398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478366c-d968-46b7-9d93-084f897d6883 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:46.169789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:46.170509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.170914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.171221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.171941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:46.172315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:46.174374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:46.191229 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1093/2186] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:46.196014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:46.222470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bb3925-c585-4112-9b84-28a48833d7da None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:46.222470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:46.236205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.238326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.241632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.244566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:46.253240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:46.323233 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1094/2187] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:46 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:46.336819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:46.354249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85b54d-fec3-4f80-81c7-179ff3cb4041 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:46.508043 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:46.509013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.509484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.509951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.510781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:46.513140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:46.558718 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1094/2188] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:46 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:46.563275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:46.581692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e62d5-a7c1-4ba6-8bac-69e694c3c442 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:46.599690 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:46.603955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.605969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.606258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.608929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:46.614106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:46.640394 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1095/2189] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:46.644523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:46.660518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f374b5a3-f7aa-4370-a100-4c5113f62c09 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:46.686947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdfa54-4168-4d9f-8f9c-a7677138c698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:46.687692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdfa54-4168-4d9f-8f9c-a7677138c698 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.688097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdfa54-4168-4d9f-8f9c-a7677138c698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.688679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdfa54-4168-4d9f-8f9c-a7677138c698 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.689422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdfa54-4168-4d9f-8f9c-a7677138c698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:46.691350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdfa54-4168-4d9f-8f9c-a7677138c698 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:46.730248 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1095/2190] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:46.734255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:46.768017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811daa7-60bf-416d-a718-9cf2dd9cc8a4 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:46.786969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:46.787694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.787967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.788232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] PATH_INFO: `/v3/projects/35360e1fdb894631b688a03713b5961d/users/aebd2ed5a07c4c67a227086db177e5c1/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.804180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:46.840377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35360e1fdb894631b688a03713b5961d, user_id=aebd2ed5a07c4c67a227086db177e5c1, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:46.859228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:46.880721 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1096/2191] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:46 2022] PUT /identity/v3/projects/35360e1fdb894631b688a03713b5961d/users/aebd2ed5a07c4c67a227086db177e5c1/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:46.891864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:46.892374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:46.892374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:46.951757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] MFA Rules not processed for user `aebd2ed5a07c4c67a227086db177e5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:46.953011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:46.995392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:47.089641 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1096/2192] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:47.105322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:47.107952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:47.187897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52ea7f-0dd2-4106-8bb0-444d859abca3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:47.194040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b152ee-b820-4617-a545-e76f2a3d16ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:47.194871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b152ee-b820-4617-a545-e76f2a3d16ad service glance] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:47.195278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b152ee-b820-4617-a545-e76f2a3d16ad service glance] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:47.196477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b152ee-b820-4617-a545-e76f2a3d16ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:47.212657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b152ee-b820-4617-a545-e76f2a3d16ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:47.214686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b152ee-b820-4617-a545-e76f2a3d16ad service glance] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:47.238208 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1097/2193] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:51.612375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:51.615939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:51.710269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ff983-c586-40f9-80e8-94c7de6bdb80 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:51.716045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:51.716990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:51.717481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:51.717747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] PATH_INFO: `/v3/users/fbcfc838ad004937b3acceac35162d25` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:51.718963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcfc838ad004937b3acceac35162d25)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:51.720997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:51.777196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcfc838ad004937b3acceac35162d25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:51.789335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcfc838ad004937b3acceac35162d25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:51.807553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcfc838ad004937b3acceac35162d25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:51.808658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcfc838ad004937b3acceac35162d25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:51.816865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcfc838ad004937b3acceac35162d25'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:51.823814 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1097/2194] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:57:51 2022] DELETE /identity/v3/users/fbcfc838ad004937b3acceac35162d25 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:51.828629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:51.829172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:51.829172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:51.854184 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:51.855565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:51.976231 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1098/2195] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:51.987109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:51.989307 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:52.068768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:52.094612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbeb29-5969-4759-969e-d46604d91243 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:52.098652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507bb74-924d-490e-a93d-be1cbea04163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:52.099336 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507bb74-924d-490e-a93d-be1cbea04163 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:52.099609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507bb74-924d-490e-a93d-be1cbea04163 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:52.099875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507bb74-924d-490e-a93d-be1cbea04163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:52.111847 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507bb74-924d-490e-a93d-be1cbea04163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:52.113518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507bb74-924d-490e-a93d-be1cbea04163 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:52.128921 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1098/2196] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 00:57:52.858967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:52.860832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:52.875559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eca72a-cca4-4a8d-ae73-e0d46d459f12 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:52.881236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:52.881961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:52.882144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:52.882415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] PATH_INFO: `/v3/projects/34e8868077dc4e70b1ecbd8c9dc7b50b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:52.883576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e8868077dc4e70b1ecbd8c9dc7b50b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:52.885601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:52.945700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e8868077dc4e70b1ecbd8c9dc7b50b', 'request_id': 'req-4d0cc29d-1d17-47e3-b506-9ec7187681a3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:52.955296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e8868077dc4e70b1ecbd8c9dc7b50b', 'request_id': 'req-4d0cc29d-1d17-47e3-b506-9ec7187681a3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:52.963624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e8868077dc4e70b1ecbd8c9dc7b50b', 'request_id': 'req-4d0cc29d-1d17-47e3-b506-9ec7187681a3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:52.977844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] The token cache is being invalidate because project 34e8868077dc4e70b1ecbd8c9dc7b50b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:57:52.978307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:52.979675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4d0cc29d-1d17-47e3-b506-9ec7187681a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:57:52.981426 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1099/2197] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:57:52 2022] DELETE /identity/v3/projects/34e8868077dc4e70b1ecbd8c9dc7b50b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:52.986328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:52.987261 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:52.987619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.013845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:53.015050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:53.111411 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1099/2198] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:57:53.117368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.117902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.118922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.167544 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:53.169263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:53.177799 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1100/2199] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:53.182818 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.208514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf0ef7-977e-46bd-bef5-47448bb487f2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.218182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.218885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.219170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.219735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.220602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:57:53.221182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:53.223660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:53.240983 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1100/2200] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:57:53.242576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.249193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:53.256196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1aa0b6-62ad-40f1-8cd5-c3ebe914083e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.262429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.263056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.266217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.266509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.267217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:53.271545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:53.331247 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1101/2201] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:57:53 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:57:53.333907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.347757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea633ba4-c94a-4315-90db-bbf9198c51c3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.436638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.437381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.437739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.438093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.439217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:53.440749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:53.465242 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1101/2202] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:57:53 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:57:53.471913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.487673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d604-526b-4462-aa09-bd630f27b42e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.492771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.493446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.493728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.493990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.494718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:53.496362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:53.508049 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1102/2203] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:57:53.512523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.525398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0356c7b-5012-4f17-a219-44e4a5c68620 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.531854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726707c6-9024-4946-8da7-0ce90eb292b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.532885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726707c6-9024-4946-8da7-0ce90eb292b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.533226 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726707c6-9024-4946-8da7-0ce90eb292b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.533562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726707c6-9024-4946-8da7-0ce90eb292b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.534593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726707c6-9024-4946-8da7-0ce90eb292b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:53.536556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726707c6-9024-4946-8da7-0ce90eb292b4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:53.554281 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1102/2204] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 00:57:53.556117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.567827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5da846-0f8e-4ae9-9301-4bf86ff2b5bd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.572922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.573604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.573895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.574163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] PATH_INFO: `/v3/projects/20163c9fb6a5490f9ad676ac84fe411b/users/6ca9626cd8bd4341ba7d26e76f34b533/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.594348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20163c9fb6a5490f9ad676ac84fe411b, user_id=6ca9626cd8bd4341ba7d26e76f34b533, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:53.597533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:53.616660 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1103/2205] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:57:53 2022] PUT /identity/v3/projects/20163c9fb6a5490f9ad676ac84fe411b/users/6ca9626cd8bd4341ba7d26e76f34b533/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:57:53.621821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371b947-c9df-4a56-88fc-6f1f172788b8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.622352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371b947-c9df-4a56-88fc-6f1f172788b8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.622852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371b947-c9df-4a56-88fc-6f1f172788b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:53.651654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1371b947-c9df-4a56-88fc-6f1f172788b8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:53.652878 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1371b947-c9df-4a56-88fc-6f1f172788b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:53.725373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:57:53.752516 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1103/2206] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:53.787131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:53.792058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:53.982502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c9001-0291-4ded-b854-a51b7c9c2846 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:53.998323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:53.998963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:53.999267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:53.999581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:54.043240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:54.043240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:54.061128 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1104/2207] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:55.518136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ae933-e80f-40a5-a674-c71c8571fdf0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:55.520572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ae933-e80f-40a5-a674-c71c8571fdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:55.520965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ae933-e80f-40a5-a674-c71c8571fdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:55.603446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-087ae933-e80f-40a5-a674-c71c8571fdf0 None None] MFA Rules not processed for user `6ca9626cd8bd4341ba7d26e76f34b533`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:55.603446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-087ae933-e80f-40a5-a674-c71c8571fdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:55.798210 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1104/2208] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:57:55.822560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:57:55.824722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:55.972775 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c2649-02ad-4ac2-ab05-a5a852bf289c service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:57:55.980610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613a34c-eb78-420c-a41e-67beb653822d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:57:55.981662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a34c-eb78-420c-a41e-67beb653822d service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:55.982370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a34c-eb78-420c-a41e-67beb653822d service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:55.982815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613a34c-eb78-420c-a41e-67beb653822d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:55.992654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613a34c-eb78-420c-a41e-67beb653822d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:57:55.994487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613a34c-eb78-420c-a41e-67beb653822d service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:57:56.016671 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1105/2209] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:57:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:57:57.531585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ef3ef-8caa-4bbb-b89c-963521ce3976 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:57:57.532180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ef3ef-8caa-4bbb-b89c-963521ce3976 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:57:57.532180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ef3ef-8caa-4bbb-b89c-963521ce3976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:57:57.556796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9c3ef3ef-8caa-4bbb-b89c-963521ce3976 None None] MFA Rules not processed for user `6ca9626cd8bd4341ba7d26e76f34b533`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:57:57.558320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9c3ef3ef-8caa-4bbb-b89c-963521ce3976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:57:57.569859 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1105/2210] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:57:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:02.794904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:02.796106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:02.797017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:02.836176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] MFA Rules not processed for user `03a3c989ecef42e084fb44d47c55b27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:02.838299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:02.905942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:02.924738 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1106/2211] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:05.779825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c9dae-dfe2-4dfc-a6d5-d18aeb83f9eb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:05.780430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c9dae-dfe2-4dfc-a6d5-d18aeb83f9eb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:05.780524 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c9dae-dfe2-4dfc-a6d5-d18aeb83f9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:05.803351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-130c9dae-dfe2-4dfc-a6d5-d18aeb83f9eb None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:05.804554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-130c9dae-dfe2-4dfc-a6d5-d18aeb83f9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:05.862546 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1106/2212] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:58:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:05.876472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:05.878210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:05.959830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe30cf-d3c3-49d1-9343-c1fc759fd7f6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:05.965654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:05.966581 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:05.966850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:05.967110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:05.977153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:05.979267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:05.988475 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1107/2213] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:58:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:58:06.738739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:06.739017 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:06.739311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:06.779651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:06.782483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:06.809585 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1107/2214] 10.222.0.23 () {58 vars in 967 bytes} [Sun Sep 11 00:58:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:06.833028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:06.843982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca4c0b-c7ef-4ce0-ba7c-9404be9cd82a service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:06.859902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:06.861339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:06.861806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:06.862202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:06.880336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:06.882924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:06.894842 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1108/2215] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:58:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:58:09.528682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:09.531946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:09.555231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a0f03-58e8-43fd-a9d1-79062086283b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:09.575645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:09.583421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:09.584189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:09.584698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] PATH_INFO: `/v3/users/398da9e11a194b9f9644dfad517d07b4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:09.588157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=398da9e11a194b9f9644dfad517d07b4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:09.590541 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:09.753639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '398da9e11a194b9f9644dfad517d07b4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:09.769302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '398da9e11a194b9f9644dfad517d07b4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:09.786869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:09.794486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '398da9e11a194b9f9644dfad517d07b4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:09.795616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '398da9e11a194b9f9644dfad517d07b4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:09.836513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5d7535de-ff14-48a7-858d-b8090a0d7b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '398da9e11a194b9f9644dfad517d07b4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:09.851503 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1108/2216] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:58:09 2022] DELETE /identity/v3/users/398da9e11a194b9f9644dfad517d07b4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:10.328817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:10.347397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141134-870b-4bc9-a304-fdaf234bd8ce service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:10.491009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:10.492037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:10.492513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:10.493041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] PATH_INFO: `/v3/projects/a84206f4832a4edda41c06ab2178c9a5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:10.494632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84206f4832a4edda41c06ab2178c9a5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:10.497149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:10.566252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84206f4832a4edda41c06ab2178c9a5', 'request_id': 'req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:10.580584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84206f4832a4edda41c06ab2178c9a5', 'request_id': 'req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:10.588696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84206f4832a4edda41c06ab2178c9a5', 'request_id': 'req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:10.603787 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] The token cache is being invalidate because project a84206f4832a4edda41c06ab2178c9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:58:10.604067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:10.605102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ecb335ae-b6b1-4bd4-bed4-53da0c7d8753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:10.606758 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1109/2217] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:58:10 2022] DELETE /identity/v3/projects/a84206f4832a4edda41c06ab2178c9a5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:10.611957 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:10.612474 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:10.612573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:10.640288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:10.642076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:10.781031 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1109/2218] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:58:10.790409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:10.791546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:10.792180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:10.822148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:10.861017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:10.864541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:10.893693 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1110/2219] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 00:58:10.895170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:10.912323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc28c5-bdb8-45d7-8c16-7f2ca164f8da None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:10.924291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:10.925478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:10.929501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:10.931777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:10.934008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:58:10.935458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:10.941833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:10.985377 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1110/2220] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:58:10.986796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:10.998468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe41dab4-3d11-42a6-9013-fdf5096915f2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:11.007018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:11.007270 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:11.007629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:11.007986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:11.008727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:11.011531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:11.082221 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1111/2221] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:58:10 2022] POST /identity/v3/projects => generated 397 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:58:11.083758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:11.123940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14516a75-bd6d-4166-af31-6beb8ced0c96 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:11.293663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:11.294901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:11.294901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:11.295635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:11.296397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:11.298917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:11.342454 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1111/2222] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:58:11 2022] POST /identity/v3/users => generated 299 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:58:11.346308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:11.364963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067ab6f-0999-429b-bdb5-1c76ba607c93 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:11.380190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:11.381187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:11.381752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:11.382201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:11.383062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:11.386744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:11.416583 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1112/2223] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:58:11.419645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:11.458322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31023881-1a15-4693-86d1-2f4f68db02fc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:11.495097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe2df4-8a86-4b69-9ff9-c234cc81966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:11.499092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe2df4-8a86-4b69-9ff9-c234cc81966a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:11.499693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe2df4-8a86-4b69-9ff9-c234cc81966a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:11.500280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe2df4-8a86-4b69-9ff9-c234cc81966a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:11.501750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe2df4-8a86-4b69-9ff9-c234cc81966a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:11.504702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe2df4-8a86-4b69-9ff9-c234cc81966a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:11.524127 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1112/2224] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:58:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:58:11.529342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:11.555785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fccc76-3b14-448e-a5b5-a8261275e76a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:11.564910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:11.567884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:11.572361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:11.572881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] PATH_INFO: `/v3/projects/79059e5c9a494116b6a681a4c1a355fe/users/85b95235d70d4acc9b91eb833957a744/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:11.598496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=79059e5c9a494116b6a681a4c1a355fe, user_id=85b95235d70d4acc9b91eb833957a744, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:11.603764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:11.642797 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1113/2225] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:58:11 2022] PUT /identity/v3/projects/79059e5c9a494116b6a681a4c1a355fe/users/85b95235d70d4acc9b91eb833957a744/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 00:58:11.655134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:11.656482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:11.656896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:11.731422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] MFA Rules not processed for user `85b95235d70d4acc9b91eb833957a744`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:11.731422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:11.822161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:11.889794 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1113/2226] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:11.902869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:11.904697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:12.012143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f705afe-85f4-4d2b-a300-e52d21b4cf3d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:12.031062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c368ea4-9b46-4ec6-9c2a-73261221c3d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:12.031902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c368ea4-9b46-4ec6-9c2a-73261221c3d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:12.032180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c368ea4-9b46-4ec6-9c2a-73261221c3d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:12.035525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c368ea4-9b46-4ec6-9c2a-73261221c3d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:12.058534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c368ea4-9b46-4ec6-9c2a-73261221c3d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:12.062849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c368ea4-9b46-4ec6-9c2a-73261221c3d1 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:12.096808 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1114/2227] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:17.735293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:17.808319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890441b-2bdf-409f-a1e2-0d4f519a94f1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:17.813155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:17.813790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:17.814055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:17.814315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] PATH_INFO: `/v3/users/85b95235d70d4acc9b91eb833957a744` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:17.815538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b95235d70d4acc9b91eb833957a744)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:17.817487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:17.874664 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b95235d70d4acc9b91eb833957a744'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:17.886141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b95235d70d4acc9b91eb833957a744'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:17.903259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b95235d70d4acc9b91eb833957a744'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:17.904216 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b95235d70d4acc9b91eb833957a744'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:17.912505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b95235d70d4acc9b91eb833957a744'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:17.924040 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1114/2228] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:58:17 2022] DELETE /identity/v3/users/85b95235d70d4acc9b91eb833957a744 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:17.927573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:17.928065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:17.928161 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:17.950328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:17.951623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:18.024903 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1115/2229] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:18.035417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:18.036887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:18.189533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebfbcd-9c65-4376-84bc-dd9b3fcb7074 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:18.194835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:18.200556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:18.200556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:18.200556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:18.227293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:18.227293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:18.243160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:18.256453 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1115/2230] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:18.956219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:18.958917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:18.999864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eae50-e35a-4925-a10e-c38f78e68433 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:19.012833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:19.013861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:19.014368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:19.015498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] PATH_INFO: `/v3/projects/79059e5c9a494116b6a681a4c1a355fe` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:19.018463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79059e5c9a494116b6a681a4c1a355fe)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:19.020778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:19.089994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:19.098138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79059e5c9a494116b6a681a4c1a355fe', 'request_id': 'req-7f86e3b1-e74f-49be-b69a-ca421d9529b1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.111018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79059e5c9a494116b6a681a4c1a355fe', 'request_id': 'req-7f86e3b1-e74f-49be-b69a-ca421d9529b1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.122478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79059e5c9a494116b6a681a4c1a355fe', 'request_id': 'req-7f86e3b1-e74f-49be-b69a-ca421d9529b1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.137942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] The token cache is being invalidate because project 79059e5c9a494116b6a681a4c1a355fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:58:19.138553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.138813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-7f86e3b1-e74f-49be-b69a-ca421d9529b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.140719 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1116/2231] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:58:18 2022] DELETE /identity/v3/projects/79059e5c9a494116b6a681a4c1a355fe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:19.340922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:19.342894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:19.363629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb75301-d572-4f7c-b17c-24884d53451a service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:19.421286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:19.421927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:19.422201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:19.422470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] PATH_INFO: `/v3/users/aebd2ed5a07c4c67a227086db177e5c1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:19.423619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aebd2ed5a07c4c67a227086db177e5c1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:19.425572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:19.480897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aebd2ed5a07c4c67a227086db177e5c1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.490952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aebd2ed5a07c4c67a227086db177e5c1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.506771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aebd2ed5a07c4c67a227086db177e5c1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.507517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aebd2ed5a07c4c67a227086db177e5c1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.514698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aebd2ed5a07c4c67a227086db177e5c1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:19.520638 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1116/2232] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:58:19 2022] DELETE /identity/v3/users/aebd2ed5a07c4c67a227086db177e5c1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:19.525587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:19.525902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:19.526179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:19.547143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:19.548861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:19.623988 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1117/2233] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:19.635686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:19.637457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:19.708139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3a4bd-a2ed-4fb6-bbbe-bc64115c0e92 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:19.712849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca5dd-9cbc-4c5c-9bf9-4b4c9b04e368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:19.714977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca5dd-9cbc-4c5c-9bf9-4b4c9b04e368 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:19.715373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca5dd-9cbc-4c5c-9bf9-4b4c9b04e368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:19.715650 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca5dd-9cbc-4c5c-9bf9-4b4c9b04e368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:19.727131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ca5dd-9cbc-4c5c-9bf9-4b4c9b04e368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:19.728702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ca5dd-9cbc-4c5c-9bf9-4b4c9b04e368 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:19.738952 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1117/2234] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:20.102191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.103926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:20.116419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d22491-96aa-4204-ad32-540871c93fa8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.121204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.121829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.122103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.122373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] PATH_INFO: `/v3/projects/35360e1fdb894631b688a03713b5961d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.123560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35360e1fdb894631b688a03713b5961d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.125504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.175432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35360e1fdb894631b688a03713b5961d', 'request_id': 'req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:20.183440 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35360e1fdb894631b688a03713b5961d', 'request_id': 'req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:20.189761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35360e1fdb894631b688a03713b5961d', 'request_id': 'req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:20.200719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] The token cache is being invalidate because project 35360e1fdb894631b688a03713b5961d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:58:20.201003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:20.201708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b39d3142-4cad-4df6-b6e1-5a8f2bc03ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:20.203355 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1118/2235] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:58:20 2022] DELETE /identity/v3/projects/35360e1fdb894631b688a03713b5961d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:20.208459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.208742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.209011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.229243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:20.230446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:20.249663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:20.296236 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1118/2236] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:20.304247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.304527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.304792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.325865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:20.327063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:20.336261 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1119/2237] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:20.340716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.350879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789c816-dfbe-4090-bbcc-000305023111 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.356735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.357335 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.357601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.357857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.358491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 00:58:20.358816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.360543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.379614 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1119/2238] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 00:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 00:58:20.384504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.394915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7bdde5-6fc2-477b-98b8-53f501aa4764 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.400523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.401125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.401417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.401658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.402289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.404233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.433315 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1120/2239] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 00:58:20 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 00:58:20.437612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.447427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de95b5-857a-491c-8511-66dcc7d64f9b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.507486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.508087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.508360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.508621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.509246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.511225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.533954 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1120/2240] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 00:58:20 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 00:58:20.538249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.549829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa4339-3396-41ea-a586-d4fe50bceacf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.554864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.555506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.555782 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.556053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.556742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.558378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.568592 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1121/2241] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:58:20.572802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.583630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50616039-f89a-44e2-a957-aa271bae4927 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.588839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b67804-6591-4d69-99fa-8f89dc6ca565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.589429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b67804-6591-4d69-99fa-8f89dc6ca565 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.589708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b67804-6591-4d69-99fa-8f89dc6ca565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.589966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b67804-6591-4d69-99fa-8f89dc6ca565 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.590601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b67804-6591-4d69-99fa-8f89dc6ca565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.592221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b67804-6591-4d69-99fa-8f89dc6ca565 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.602856 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1121/2242] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 00:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 00:58:20.607005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.617201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59aebf7-f3f0-4c13-8c96-e01daa106504 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.622274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.622913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.623211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.623489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] PATH_INFO: `/v3/projects/b57cbc69b5354618a8c17695dabfc255/users/61f2059253ed4e1c81ed23ef8faba4b1/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.632009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:20.640403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b57cbc69b5354618a8c17695dabfc255, user_id=61f2059253ed4e1c81ed23ef8faba4b1, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.643835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.658702 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1122/2243] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 00:58:20 2022] PUT /identity/v3/projects/b57cbc69b5354618a8c17695dabfc255/users/61f2059253ed4e1c81ed23ef8faba4b1/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:20.671937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.672215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.672477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.701431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] MFA Rules not processed for user `61f2059253ed4e1c81ed23ef8faba4b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:20.702647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:20.776210 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1122/2244] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:20.787629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.789851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:20.850747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4fc69-83f6-4fa2-85dc-e3d255197eae admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:20.854966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fee9c3-9702-41ad-bc04-6fda962f9396 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:20.855558 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fee9c3-9702-41ad-bc04-6fda962f9396 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:20.855829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fee9c3-9702-41ad-bc04-6fda962f9396 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:20.856096 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fee9c3-9702-41ad-bc04-6fda962f9396 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:20.865160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fee9c3-9702-41ad-bc04-6fda962f9396 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:20.866714 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fee9c3-9702-41ad-bc04-6fda962f9396 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:20.878841 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1123/2245] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:20.913410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:20.950924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:21.014793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856684e-00b0-4a7f-a320-de115972b31c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:21.026677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:21.027473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:21.027860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:21.028197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] PATH_INFO: `/v3/users/61f2059253ed4e1c81ed23ef8faba4b1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:21.029736 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f2059253ed4e1c81ed23ef8faba4b1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:21.031860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:21.141822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f2059253ed4e1c81ed23ef8faba4b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.161712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f2059253ed4e1c81ed23ef8faba4b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.183945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f2059253ed4e1c81ed23ef8faba4b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.184663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f2059253ed4e1c81ed23ef8faba4b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.191696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f2059253ed4e1c81ed23ef8faba4b1'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.199236 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1123/2246] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 00:58:20 2022] DELETE /identity/v3/users/61f2059253ed4e1c81ed23ef8faba4b1 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:21.202494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:21.202980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:21.203072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:21.223156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:21.224412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:21.295915 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1124/2247] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:21.305565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:21.307026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:21.364556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16c34d-50af-4437-a857-0e7d1c5ec36d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:21.368181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c901844-7827-49a5-81ca-e31e0d6acf7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:21.368794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c901844-7827-49a5-81ca-e31e0d6acf7c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:21.369129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c901844-7827-49a5-81ca-e31e0d6acf7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:21.369392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c901844-7827-49a5-81ca-e31e0d6acf7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:21.378540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c901844-7827-49a5-81ca-e31e0d6acf7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:21.380020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c901844-7827-49a5-81ca-e31e0d6acf7c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:21.390008 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1124/2248] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:21.702376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:21.703907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:21.718579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b91a8-3ca6-4174-a091-6c5a09a84af8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:21.723883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:21.724527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:21.724800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:21.725069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] PATH_INFO: `/v3/projects/b57cbc69b5354618a8c17695dabfc255` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:21.726279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b57cbc69b5354618a8c17695dabfc255)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:21.732070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:21.786733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b57cbc69b5354618a8c17695dabfc255', 'request_id': 'req-aa5e5c80-cf81-4868-b48d-e58134abf7fa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.796155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b57cbc69b5354618a8c17695dabfc255', 'request_id': 'req-aa5e5c80-cf81-4868-b48d-e58134abf7fa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.802672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b57cbc69b5354618a8c17695dabfc255', 'request_id': 'req-aa5e5c80-cf81-4868-b48d-e58134abf7fa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.813427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] The token cache is being invalidate because project b57cbc69b5354618a8c17695dabfc255 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 00:58:21.813705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.814381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aa5e5c80-cf81-4868-b48d-e58134abf7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 00:58:21.815977 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1125/2249] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 00:58:21 2022] DELETE /identity/v3/projects/b57cbc69b5354618a8c17695dabfc255 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 00:58:26.646510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d97b-524e-4940-b74c-e28050667abc None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:26.647073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d97b-524e-4940-b74c-e28050667abc None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:26.647073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5d97b-524e-4940-b74c-e28050667abc None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:26.649502 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1125/2250] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:58:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:58:26.656174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46beec5-3401-4124-bc6d-7f4a4e7aa195 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:26.656683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46beec5-3401-4124-bc6d-7f4a4e7aa195 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:26.656797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46beec5-3401-4124-bc6d-7f4a4e7aa195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:26.679419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f46beec5-3401-4124-bc6d-7f4a4e7aa195 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:26.680649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f46beec5-3401-4124-bc6d-7f4a4e7aa195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:26.690369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:26.753816 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1126/2251] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:58:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:26.765634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d97b-524e-4940-b74c-e28050667abc None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:26.767530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-35b5d97b-524e-4940-b74c-e28050667abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:26.826238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:26.850490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5d97b-524e-4940-b74c-e28050667abc None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:26.854202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8b906-396f-494e-9bb4-b270a3c5751b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:26.854810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8b906-396f-494e-9bb4-b270a3c5751b service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:26.855077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8b906-396f-494e-9bb4-b270a3c5751b service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:26.855375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8b906-396f-494e-9bb4-b270a3c5751b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:26.866755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8b906-396f-494e-9bb4-b270a3c5751b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:26.868350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8b906-396f-494e-9bb4-b270a3c5751b service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:26.885596 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1126/2252] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:26.966978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993639-2900-44fd-9a84-cb1cd8cfda7c None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:26.967285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993639-2900-44fd-9a84-cb1cd8cfda7c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:26.967562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29993639-2900-44fd-9a84-cb1cd8cfda7c None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:26.968923 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1127/2253] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:58:26.975336 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc80f-d067-4947-ac7c-d9ded073d186 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:26.975620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc80f-d067-4947-ac7c-d9ded073d186 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:26.975918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050dc80f-d067-4947-ac7c-d9ded073d186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:26.996303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-050dc80f-d067-4947-ac7c-d9ded073d186 None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:26.997508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-050dc80f-d067-4947-ac7c-d9ded073d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:27.049458 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1127/2254] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:58:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:27.061534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993639-2900-44fd-9a84-cb1cd8cfda7c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:27.063825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-29993639-2900-44fd-9a84-cb1cd8cfda7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:27.084437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29993639-2900-44fd-9a84-cb1cd8cfda7c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:27.089384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9bb135-481f-4796-bbea-854a34c3880b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:27.089993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb135-481f-4796-bbea-854a34c3880b service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:27.090247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb135-481f-4796-bbea-854a34c3880b service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:27.090501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9bb135-481f-4796-bbea-854a34c3880b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:27.097857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bb135-481f-4796-bbea-854a34c3880b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:27.099368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9bb135-481f-4796-bbea-854a34c3880b service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:27.107266 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1128/2255] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:27.423385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f36339-38ae-4710-9d4c-d3d61fc6a2b3 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:27.423673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f36339-38ae-4710-9d4c-d3d61fc6a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:27.423960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f36339-38ae-4710-9d4c-d3d61fc6a2b3 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:27.425295 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1128/2256] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 00:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 00:58:27.430939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc70b6-f2fa-4c16-b6f0-d0ba4c57c011 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:27.431241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc70b6-f2fa-4c16-b6f0-d0ba4c57c011 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:27.431513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc70b6-f2fa-4c16-b6f0-d0ba4c57c011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:27.452295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7edc70b6-f2fa-4c16-b6f0-d0ba4c57c011 None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:27.453504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7edc70b6-f2fa-4c16-b6f0-d0ba4c57c011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:27.464064 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1129/2257] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 00:58:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:27.474752 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f36339-38ae-4710-9d4c-d3d61fc6a2b3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:27.486862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f36339-38ae-4710-9d4c-d3d61fc6a2b3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:27.491269 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:27.491815 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:27.492085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:27.492346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:27.503935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:27.505366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:27.516643 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1129/2258] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:30.543127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:30.543637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:30.543752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:30.563610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:30.564808 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:30.622239 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1130/2259] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 00:58:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:30.634585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:30.636232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:30.709081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b95f3d-0503-41fb-90b6-fb9fcf3c2efd service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:30.715595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1d250-36c0-49eb-8487-cea21de0617f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:30.716197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1d250-36c0-49eb-8487-cea21de0617f service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:30.716468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1d250-36c0-49eb-8487-cea21de0617f service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:30.716728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1d250-36c0-49eb-8487-cea21de0617f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:30.725951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1d250-36c0-49eb-8487-cea21de0617f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:30.727587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1d250-36c0-49eb-8487-cea21de0617f service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:30.737321 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1130/2260] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:58:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:58:40.615362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=59 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:40.733273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=69 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:58:45.776112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:45.777991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:45.852559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dacf506-a9fa-4d59-adf3-3178f2a33106 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:45.857671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:45.858312 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:45.858586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:45.858848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:45.860675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:45.873307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:45.874902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:45.885231 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1131/2261] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:49.063042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abedca-ec19-4349-aa02-c15890eab47c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:49.063713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abedca-ec19-4349-aa02-c15890eab47c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:49.063713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91abedca-ec19-4349-aa02-c15890eab47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:49.084942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-91abedca-ec19-4349-aa02-c15890eab47c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:49.086166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-91abedca-ec19-4349-aa02-c15890eab47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:49.155783 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1131/2262] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:49.192987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:49.194411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:49.209254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abbc42-7575-4c76-b321-4ef5bc4f67b2 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:49.214898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:49.215553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:49.215830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:49.216092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:49.224571 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:49.226120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:49.236387 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1132/2263] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:53.139814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18722d6-4ee3-4081-89ec-9f68cb2e5339 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:53.140422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18722d6-4ee3-4081-89ec-9f68cb2e5339 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:53.141168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18722d6-4ee3-4081-89ec-9f68cb2e5339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:53.162480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b18722d6-4ee3-4081-89ec-9f68cb2e5339 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:53.163730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b18722d6-4ee3-4081-89ec-9f68cb2e5339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:53.173122 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1132/2264] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:58:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:53.200994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:53.213241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04dbf3-fb3a-4c9e-b81c-c7f2ed63a63c service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:53.220409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:53.220992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:53.221262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:53.221530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:53.230471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:53.232094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:53.241902 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1133/2265] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:58:53.581714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736e8fe-40a1-48a4-9b68-9ccdfe3bb3bb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:53.582001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736e8fe-40a1-48a4-9b68-9ccdfe3bb3bb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:53.582268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736e8fe-40a1-48a4-9b68-9ccdfe3bb3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:53.602713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1736e8fe-40a1-48a4-9b68-9ccdfe3bb3bb None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:53.603945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1736e8fe-40a1-48a4-9b68-9ccdfe3bb3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:53.612973 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1133/2266] 10.222.0.23 () {58 vars in 967 bytes} [Sun Sep 11 00:58:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:53.636175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:53.648124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa360f2-f153-4a79-a25c-4e9eba8fbeef service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:53.654488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:53.655091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:53.655393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:53.655655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:53.664804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:53.666991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:53.675720 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1134/2267] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 00:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 00:58:57.469737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9db6f-a614-4a13-8699-b330cd791bc0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:57.470639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9db6f-a614-4a13-8699-b330cd791bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:57.470639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c9db6f-a614-4a13-8699-b330cd791bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:57.496565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b1c9db6f-a614-4a13-8699-b330cd791bc0 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:58:57.497800 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b1c9db6f-a614-4a13-8699-b330cd791bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:58:57.507214 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1134/2268] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:58:57.533735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:58:57.546673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae55a88-716c-4e17-a507-20634ec87e12 service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:58:57.553485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:58:57.554080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:58:57.554400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:58:57.554683 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:58:57.563432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:58:57.565000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:58:57.575372 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1135/2269] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:59:00.484108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ee541-fcd2-458b-8ef7-4100a2de24e6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:00.484740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ee541-fcd2-458b-8ef7-4100a2de24e6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:00.484740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ee541-fcd2-458b-8ef7-4100a2de24e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:00.513412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-214ee541-fcd2-458b-8ef7-4100a2de24e6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:00.514606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-214ee541-fcd2-458b-8ef7-4100a2de24e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:00.524129 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1135/2270] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:00.559658 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:59:00.561244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:00.580145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59250de6-41ff-4a4a-bf2c-e731172d3128 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:59:00.587343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:59:00.587944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:00.588213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:00.588465 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:00.597639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:59:00.599374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:59:00.609749 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1136/2271] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:59:04.723061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cca9a7-8c19-4a5b-9ac4-3a143f45a811 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:04.723718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cca9a7-8c19-4a5b-9ac4-3a143f45a811 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:04.723920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cca9a7-8c19-4a5b-9ac4-3a143f45a811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:04.746411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c6cca9a7-8c19-4a5b-9ac4-3a143f45a811 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:04.748100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c6cca9a7-8c19-4a5b-9ac4-3a143f45a811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:04.757929 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1136/2272] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:04.787704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:59:04.800719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49363c-c34f-4cfa-a66b-926e96eaa8d5 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:59:04.809106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:59:04.810222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:04.810570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:04.811054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:04.822302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:59:04.824177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:59:04.836968 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1137/2273] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:59:07.897340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afcf13-b17f-4968-a8bd-a9cecf6ae767 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:07.897789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afcf13-b17f-4968-a8bd-a9cecf6ae767 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:07.897887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afcf13-b17f-4968-a8bd-a9cecf6ae767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:07.918542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-15afcf13-b17f-4968-a8bd-a9cecf6ae767 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:07.919868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-15afcf13-b17f-4968-a8bd-a9cecf6ae767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:07.929517 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1137/2274] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:07.968692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:59:07.982485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280e060-d588-4b5a-977e-0177dfb969e8 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:59:07.989553 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5d48f-46a8-4ded-ab5a-27191b97569f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:59:07.990151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5d48f-46a8-4ded-ab5a-27191b97569f service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:07.990448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5d48f-46a8-4ded-ab5a-27191b97569f service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:07.990716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5d48f-46a8-4ded-ab5a-27191b97569f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:07.999230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5d48f-46a8-4ded-ab5a-27191b97569f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:59:08.000804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5d48f-46a8-4ded-ab5a-27191b97569f service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:59:08.010483 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1138/2275] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:59:15.908227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520137b-d7ca-43c4-8375-aefbcd38d850 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:15.908851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520137b-d7ca-43c4-8375-aefbcd38d850 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:15.908851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520137b-d7ca-43c4-8375-aefbcd38d850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:15.916642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=46 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:15.930098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d520137b-d7ca-43c4-8375-aefbcd38d850 None None] MFA Rules not processed for user `364cf4a7a3bf4e70973c78a5d690e998`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:15.931440 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d520137b-d7ca-43c4-8375-aefbcd38d850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:15.989794 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1138/2276] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:59:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:18.006419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=74 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:25.982908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=24 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:27.836068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19926c-306f-4b91-bb29-2250446e438b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:27.836610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19926c-306f-4b91-bb29-2250446e438b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:27.836610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19926c-306f-4b91-bb29-2250446e438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:27.857487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bb19926c-306f-4b91-bb29-2250446e438b None None] MFA Rules not processed for user `6ca9626cd8bd4341ba7d26e76f34b533`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:27.858716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bb19926c-306f-4b91-bb29-2250446e438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:27.912916 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1139/2277] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:59:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:37.906177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=25 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 00:59:45.109917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:45.110538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:45.110538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:45.136472 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] MFA Rules not processed for user `03a3c989ecef42e084fb44d47c55b27e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:45.137711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:45.204718 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1139/2278] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:59:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:47.613198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa29217-f74d-4060-ad3e-72d869630192 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:47.613198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa29217-f74d-4060-ad3e-72d869630192 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:47.613198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa29217-f74d-4060-ad3e-72d869630192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:47.633175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4fa29217-f74d-4060-ad3e-72d869630192 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:47.634483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fa29217-f74d-4060-ad3e-72d869630192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:47.642411 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1140/2279] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:47.664116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:59:47.682398 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3407c68-25fb-4878-b1bf-9eb9a0cdab09 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:59:47.690006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:59:47.690636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:47.691163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:47.691496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:47.703734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:59:47.705460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:59:47.719354 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1140/2280] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:59:53.027786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31f902-234e-4204-a594-71f02dfb7390 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:53.028342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31f902-234e-4204-a594-71f02dfb7390 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:53.028342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31f902-234e-4204-a594-71f02dfb7390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:53.049046 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9f31f902-234e-4204-a594-71f02dfb7390 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:53.050255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9f31f902-234e-4204-a594-71f02dfb7390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:53.059082 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1141/2281] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:53.080712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:59:53.093848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401b039-198e-414e-a215-cb61e6d33776 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:59:53.101274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30460045-f8d0-4257-bf11-f2e5588bd923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:59:53.102181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30460045-f8d0-4257-bf11-f2e5588bd923 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:53.102454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30460045-f8d0-4257-bf11-f2e5588bd923 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:53.102713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30460045-f8d0-4257-bf11-f2e5588bd923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:53.111357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30460045-f8d0-4257-bf11-f2e5588bd923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:59:53.112927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30460045-f8d0-4257-bf11-f2e5588bd923 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:59:53.124676 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1141/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 00:59:54.102459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:54.102987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:54.102987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:54.123226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] MFA Rules not processed for user `44c365d540894a908b34fe51d854e96e`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:54.124423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:54.184899 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1142/2283] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 00:59:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:56.084641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d50a6e-f01d-4d1c-9076-3dec1c8ae34c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:56.085229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d50a6e-f01d-4d1c-9076-3dec1c8ae34c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:56.085229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d50a6e-f01d-4d1c-9076-3dec1c8ae34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:56.106677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-92d50a6e-f01d-4d1c-9076-3dec1c8ae34c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 00:59:56.107923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-92d50a6e-f01d-4d1c-9076-3dec1c8ae34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 00:59:56.117496 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1142/2284] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 00:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 00:59:56.147314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 00:59:56.161773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732e92f-cd2f-430b-81e9-ea1770a4642e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 00:59:56.169413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 00:59:56.170139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 00:59:56.170414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 00:59:56.170657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 00:59:56.180037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 00:59:56.181604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 00:59:56.191297 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1143/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 00:59:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:01.358979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e3eac-f2fe-49fa-bb01-7ca79484d79b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:01.359561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e3eac-f2fe-49fa-bb01-7ca79484d79b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:01.359561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e3eac-f2fe-49fa-bb01-7ca79484d79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:01.381482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d51e3eac-f2fe-49fa-bb01-7ca79484d79b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:01.382708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d51e3eac-f2fe-49fa-bb01-7ca79484d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:01.391904 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1143/2286] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 01:00:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:01.417925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:01.431601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d33ff9-7c96-49be-9822-8e62ed5848a1 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:01.439823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:01.440523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:01.440998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:01.441336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:01.451980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:01.453911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:01.467669 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1144/2287] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:04.544563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac7a43-a0fc-4896-a38f-7fdfa2bf77c0 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:04.545137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac7a43-a0fc-4896-a38f-7fdfa2bf77c0 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:04.545137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac7a43-a0fc-4896-a38f-7fdfa2bf77c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:04.565117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-28ac7a43-a0fc-4896-a38f-7fdfa2bf77c0 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:04.566236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-28ac7a43-a0fc-4896-a38f-7fdfa2bf77c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:04.574585 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1144/2288] 10.222.0.46 () {58 vars in 1016 bytes} [Sun Sep 11 01:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:04.598191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:04.609513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be3f3c-8d18-488b-9b3e-8e2cd88e0186 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:04.615944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab31da-70f9-4121-a969-4e3aee80f2e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:04.616602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab31da-70f9-4121-a969-4e3aee80f2e1 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:04.616862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab31da-70f9-4121-a969-4e3aee80f2e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:04.617061 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab31da-70f9-4121-a969-4e3aee80f2e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:04.625657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab31da-70f9-4121-a969-4e3aee80f2e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:04.627260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab31da-70f9-4121-a969-4e3aee80f2e1 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:04.636918 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1145/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:14.559821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=46 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:14.632772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=49 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:23.259863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:23.260422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:23.260422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:23.279783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] MFA Rules not processed for user `081d365c7b15443aa76c7da2ca444a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:23.281016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:23.346931 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1145/2290] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:31.131328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:31.131909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:31.131909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:31.153120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] MFA Rules not processed for user `364cf4a7a3bf4e70973c78a5d690e998`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:31.154633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:31.162813 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1146/2291] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:33.338728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=25 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:35.334804 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:35.336394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:35.348924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec22ac-4fce-40ad-887d-1b7753f00754 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:35.353048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:35.353873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:35.354058 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:35.354592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:35.356747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:35.371938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:35.373925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:35.386374 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1146/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:38.030170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:38.041193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1baf88-d11c-4537-9cb3-8a584c5f1861 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:38.045926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:38.046649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:38.046994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:38.047383 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:38.049605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:38.065401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:38.066862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:38.084160 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1147/2293] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:39.277205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:39.279095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:39.295010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441e783-e508-4bd7-87f8-a5c56718d1b8 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:39.302801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:39.303508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:39.303757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:39.304045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] PATH_INFO: `/v3/users/6ca9626cd8bd4341ba7d26e76f34b533` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:39.305203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ca9626cd8bd4341ba7d26e76f34b533)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:39.307210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:39.363400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ca9626cd8bd4341ba7d26e76f34b533'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.372764 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ca9626cd8bd4341ba7d26e76f34b533'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.389945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ca9626cd8bd4341ba7d26e76f34b533'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.390707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ca9626cd8bd4341ba7d26e76f34b533'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.399544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9ff8e973-0253-46ee-ac60-67cb0554f765 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ca9626cd8bd4341ba7d26e76f34b533'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.406749 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1147/2294] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:00:39 2022] DELETE /identity/v3/users/6ca9626cd8bd4341ba7d26e76f34b533 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:00:39.704458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:39.718604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4e4fb-e43b-43bd-8449-6904d089b4fd service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:39.797361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:39.798149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:39.798422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:39.798862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] PATH_INFO: `/v3/projects/20163c9fb6a5490f9ad676ac84fe411b` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:39.804860 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20163c9fb6a5490f9ad676ac84fe411b)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:39.807084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:39.884950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20163c9fb6a5490f9ad676ac84fe411b', 'request_id': 'req-dd494577-5e9d-4192-8ba3-757501207c85'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.894889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20163c9fb6a5490f9ad676ac84fe411b', 'request_id': 'req-dd494577-5e9d-4192-8ba3-757501207c85'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.906154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20163c9fb6a5490f9ad676ac84fe411b', 'request_id': 'req-dd494577-5e9d-4192-8ba3-757501207c85'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.928284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] The token cache is being invalidate because project 20163c9fb6a5490f9ad676ac84fe411b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:00:39.928618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.929289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-dd494577-5e9d-4192-8ba3-757501207c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:00:39.931948 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1148/2295] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:00:39 2022] DELETE /identity/v3/projects/20163c9fb6a5490f9ad676ac84fe411b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:00:39.937982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:39.938334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:39.938711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:39.961050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:39.963769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:40.053993 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1148/2296] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:40.061948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.062224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.062490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.083800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:40.085051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:40.094626 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1149/2297] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:40.098798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.110041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164a804-8e75-444a-9f5c-358d7acdd7f8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.115138 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.115797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.116070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.116330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.116974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:00:40.117274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.118875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.133071 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1149/2298] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:00:40.137556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.149230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97351b00-4c12-4921-a84d-5c47c7a38ae5 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.159063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.159750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.160023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.160288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.161301 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.162889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.197914 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1150/2299] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:00:40 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:00:40.202635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.214344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cafbda-c364-47cc-b605-cda2c7e9b741 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.271581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:40.292807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.293548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.293693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.294377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.295347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.297621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.326130 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1150/2300] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:00:40 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:00:40.330498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.341713 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be246684-14fd-445d-9a8c-3576ece7031b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.347369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.348173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.348589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.349349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.350243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.351952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.364246 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1151/2301] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:00:40.368601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.380114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cc098-c755-4921-bb31-a72db6f5735a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.385677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1f7d8-e2b6-4ff9-a5ad-5cf8191023ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.386828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1f7d8-e2b6-4ff9-a5ad-5cf8191023ef admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.387361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1f7d8-e2b6-4ff9-a5ad-5cf8191023ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.387812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1f7d8-e2b6-4ff9-a5ad-5cf8191023ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.388784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1f7d8-e2b6-4ff9-a5ad-5cf8191023ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.390683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1f7d8-e2b6-4ff9-a5ad-5cf8191023ef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.403488 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1151/2302] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:00:40.407959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.423017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c369-df7e-4de3-aa0b-757eef91363b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.428530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.429395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.429679 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.429977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] PATH_INFO: `/v3/projects/69a25da8ed8849bba71977e48ec7bd8d/users/b0ba6f92b3d34bbeb1f7d52de32cd29c/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.485746 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a25da8ed8849bba71977e48ec7bd8d, user_id=b0ba6f92b3d34bbeb1f7d52de32cd29c, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.491648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.521010 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1152/2303] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:00:40 2022] PUT /identity/v3/projects/69a25da8ed8849bba71977e48ec7bd8d/users/b0ba6f92b3d34bbeb1f7d52de32cd29c/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:00:40.526966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121789ee-0477-47cc-836a-bf9f79ee2c9e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.527478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121789ee-0477-47cc-836a-bf9f79ee2c9e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.528230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121789ee-0477-47cc-836a-bf9f79ee2c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.563726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-121789ee-0477-47cc-836a-bf9f79ee2c9e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:40.566286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-121789ee-0477-47cc-836a-bf9f79ee2c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:40.645026 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1152/2304] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:40.663621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:40.665848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:40.676496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:40.747106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53eaa5-9572-439e-bab5-bac495591710 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:40.751970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:40.752599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:40.753030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:40.753307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:40.763964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:40.765438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:40.775826 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1153/2305] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:46.002176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.002695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.002695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.043871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] MFA Rules not processed for user `b0ba6f92b3d34bbeb1f7d52de32cd29c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:46.045779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:46.119456 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1153/2306] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:46.124332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:46.134181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef2e24b-5879-4f10-abb3-e728a75abc12 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:46.139169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:46.139825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.140099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.140368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.141017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:46.142947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:46.173127 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1154/2307] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:00:46 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:00:46.177402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:46.195152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8f0b0-1e3f-4e00-968f-c25b8dfe9913 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:46.251434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:46.261846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:46.264036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.264489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.265416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.266517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:46.270828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:46.306489 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1154/2308] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:00:46 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:00:46.310615 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:46.321828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ace43d-3072-41ae-bbc6-04e5077f8268 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:46.326989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:46.327692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.327968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.328233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.328906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:46.330527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:46.342307 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1155/2309] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:00:46.346580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:46.357290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0033d59-5fb6-43ce-8f5c-9a70706a23d2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:46.362109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:46.362839 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.363114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.363430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] PATH_INFO: `/v3/projects/1b367e6b2c73466cbb5b25183fbee53a/users/d4e9b26a95ec451eaed6c41385f44957/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.378459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b367e6b2c73466cbb5b25183fbee53a, user_id=d4e9b26a95ec451eaed6c41385f44957, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:46.381082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:46.396727 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1155/2310] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:00:46 2022] PUT /identity/v3/projects/1b367e6b2c73466cbb5b25183fbee53a/users/d4e9b26a95ec451eaed6c41385f44957/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:00:46.400861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:46.413421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13611ba-b58d-4df3-802c-4cb4f04602d5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:46.484765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97749136-493e-4ab9-9dee-ea8ede955a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:46.485478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97749136-493e-4ab9-9dee-ea8ede955a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.485818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97749136-493e-4ab9-9dee-ea8ede955a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.486099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97749136-493e-4ab9-9dee-ea8ede955a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.486849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97749136-493e-4ab9-9dee-ea8ede955a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:46.489107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97749136-493e-4ab9-9dee-ea8ede955a90 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:46.500378 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1156/2311] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:00:46.504538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:46.516990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99031b-7749-4567-9250-56fe3831396d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:46.522776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:46.523607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:46.524065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:46.524510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] PATH_INFO: `/v3/projects/1b367e6b2c73466cbb5b25183fbee53a/users/d4e9b26a95ec451eaed6c41385f44957/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:46.527079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b367e6b2c73466cbb5b25183fbee53a, user_id=d4e9b26a95ec451eaed6c41385f44957, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:46.530593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:46.556039 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1156/2312] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:00:46 2022] PUT /identity/v3/projects/1b367e6b2c73466cbb5b25183fbee53a/users/d4e9b26a95ec451eaed6c41385f44957/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:00:52.634619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70b2b-52cb-4b2a-8512-f1f2763ed090 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:52.635136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70b2b-52cb-4b2a-8512-f1f2763ed090 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:52.635136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70b2b-52cb-4b2a-8512-f1f2763ed090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:52.721652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8eb70b2b-52cb-4b2a-8512-f1f2763ed090 None None] MFA Rules not processed for user `d4e9b26a95ec451eaed6c41385f44957`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:52.723218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8eb70b2b-52cb-4b2a-8512-f1f2763ed090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:52.849543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:52.874736 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1157/2313] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:00:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:52.898122 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:52.901500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:52.992624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c480f1c-fa60-499a-8be2-a095b8ac3c1a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:52.997791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:52.998401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:52.998842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:52.999298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:53.072774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:53.074427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:53.097694 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1157/2314] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:53.252400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:53.252400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:53.252400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:53.273773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:00:53.275061 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:53.339657 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1158/2315] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 01:00:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:00:53.403797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:53.405443 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:53.497798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:00:53.503656 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bb6451-9f4c-40e1-b0b4-d260ac494207 service glance] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:53.512174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:53.512174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:53.512174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:53.512605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:53.521611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:53.523282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:53.532818 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1158/2316] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:00:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 01:00:53.621866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:53.623996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:53.733132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e22e68-6ff4-4b0b-9acc-2bf61a1f0f1c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:53.744316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:53.745552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:53.746169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:53.746761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:53.753056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:53.894685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:53.899991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:53.919813 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1159/2317] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:53 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:00:56.174317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:00:56.195553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462da6b5-41c1-4af8-9045-86ccf5911bde service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:00:56.205391 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:00:56.206107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:00:56.206489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:00:56.206821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:00:56.213324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:00:56.324251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:00:56.326142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530800b-a063-4699-b1fa-701b9472b87d service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:00:56.343235 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1159/2318] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:00:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:00.535508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:00.584776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:00.600646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabee4e-902d-42f5-86b3-07d4c31fbdf0 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:00.606774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:00.607540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:00.607868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:00.608212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:00.610012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:00.682495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:00.684605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:00.698696 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1160/2319] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:03.397869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:03.398418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:03.398418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:03.418894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] MFA Rules not processed for user `b0ba6f92b3d34bbeb1f7d52de32cd29c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:03.420346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:03.428030 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1160/2320] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:03.820339 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:03.821888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:03.833727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e7b08-b94d-4988-a4c4-5d1520ff7d4b service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:03.838121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:03.838904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:03.839235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:03.839547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:03.841564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:03.852233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:03.853657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:03.863007 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1161/2321] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:04.452862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:04.454626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:04.530607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daee8b9-047d-4b4a-8ab6-cdc2580e6a49 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:04.545285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:04.545988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:04.546305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:04.546518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:04.548962 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:04.565739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:04.565739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:04.581629 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1161/2322] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:04.645144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:04.646564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:04.664666 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7568d-d364-425e-a222-56431ec60f0f service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:04.670646 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:04.671815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:04.672434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:04.672949 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:04.675048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:04.690170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:04.692929 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:04.712812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:04.714369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:04.715387 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1162/2323] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:04.733126 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28b336-068a-4a6a-a8ce-71dc00d14256 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:04.747015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:04.747734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:04.748014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:04.748279 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] PATH_INFO: `/v3/users/03a3c989ecef42e084fb44d47c55b27e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:04.749766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a3c989ecef42e084fb44d47c55b27e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:04.751805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:04.801177 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a3c989ecef42e084fb44d47c55b27e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:04.810894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a3c989ecef42e084fb44d47c55b27e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:04.824317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:04.827201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a3c989ecef42e084fb44d47c55b27e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:04.828046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a3c989ecef42e084fb44d47c55b27e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:04.837707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-507b9b87-7e56-41b9-8fcd-57b5f14cfcef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a3c989ecef42e084fb44d47c55b27e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:04.845388 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1162/2324] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:04 2022] DELETE /identity/v3/users/03a3c989ecef42e084fb44d47c55b27e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:05.025910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.039079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d750e8-2420-422b-9e52-03e03840d358 service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.099972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.100619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.100924 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.101191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] PATH_INFO: `/v3/projects/ac8c0850e1ac43e7913a88dbb7b7e9f8` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.102319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8c0850e1ac43e7913a88dbb7b7e9f8)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.104311 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.158572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8c0850e1ac43e7913a88dbb7b7e9f8', 'request_id': 'req-e1510665-ffb2-4358-ab16-5585d1446b4a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:05.166759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8c0850e1ac43e7913a88dbb7b7e9f8', 'request_id': 'req-e1510665-ffb2-4358-ab16-5585d1446b4a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:05.173048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8c0850e1ac43e7913a88dbb7b7e9f8', 'request_id': 'req-e1510665-ffb2-4358-ab16-5585d1446b4a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:05.176848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:05.184425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] The token cache is being invalidate because project ac8c0850e1ac43e7913a88dbb7b7e9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:05.184878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:05.186001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e1510665-ffb2-4358-ab16-5585d1446b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:05.187656 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1163/2325] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:05 2022] DELETE /identity/v3/projects/ac8c0850e1ac43e7913a88dbb7b7e9f8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:05.192713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.192990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.193258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.213525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:05.214781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:05.284546 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1163/2326] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:01:05.292639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.292917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.293209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.316923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:05.318295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:05.328771 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1164/2327] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:05.333141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.343882 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389edf7a-e1df-4f0a-87ae-d0db58086190 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.350501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.351896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.353121 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.353529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.354353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:05.354658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.356582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.370019 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1164/2328] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:01:05.374145 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.385519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff2eea-cb92-4319-a367-f09668694fa7 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.390374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.390991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.391300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.391571 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.392233 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.394135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.425596 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1165/2329] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:05.429723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.440631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed34d32-579d-4b0c-a290-3629b9fa41cc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.496658 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.497324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.497517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.497783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.498416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.500374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.524162 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1165/2330] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:05.528663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.539752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ff468-c99a-47b8-b7dd-aa47995fa42d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.545469 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.546066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.546344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.546601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.547368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.548970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.559637 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1166/2331] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:01:05.563557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.574459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19017c-4834-4a68-807a-3984d648a435 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.580454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29060d59-64d5-4964-a4b4-4a000c9ddd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.581136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29060d59-64d5-4964-a4b4-4a000c9ddd37 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.581513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29060d59-64d5-4964-a4b4-4a000c9ddd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.581806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29060d59-64d5-4964-a4b4-4a000c9ddd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.582458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29060d59-64d5-4964-a4b4-4a000c9ddd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.584109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29060d59-64d5-4964-a4b4-4a000c9ddd37 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.595403 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1166/2332] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:05.599704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.609874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc6993-0239-4cf6-b461-ff55ccdffb57 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.614917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.615611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.615886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.616156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] PATH_INFO: `/v3/projects/13d72aac78d5408991fa978fee19502e/users/c838d7c6c7fe4bdb82581cf592c30bfa/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.631815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d72aac78d5408991fa978fee19502e, user_id=c838d7c6c7fe4bdb82581cf592c30bfa, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.634438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.649613 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1167/2333] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:05 2022] PUT /identity/v3/projects/13d72aac78d5408991fa978fee19502e/users/c838d7c6c7fe4bdb82581cf592c30bfa/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:05.660080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.660367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.660638 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.692410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] MFA Rules not processed for user `c838d7c6c7fe4bdb82581cf592c30bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:05.693648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:05.738651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:05.767435 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1167/2334] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:05.772232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.834473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02c595-0fa7-4683-b7d9-95bc19b535ad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.839327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.840133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.840603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.840603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.841251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.843138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.876576 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1168/2335] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:05.877495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:05.889966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4aa32c-5a50-4966-9fc9-6b64f29d78c1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:05.958205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:05.958550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:05.958917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:05.959306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:05.960032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:05.962284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:05.991838 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1168/2336] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:05 2022] POST /identity/v3/users => generated 308 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:05.993522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.004745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d330c77-a980-4537-bd94-7c188e7eef0c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.010987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.011722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.012068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.012386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.013100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.014773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.030349 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1169/2337] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:06.033202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.044286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea71c0d-491b-4b3b-bb33-0710ed1d6f3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.049208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d557d-55df-4360-b471-48e37b1265d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.049863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d557d-55df-4360-b471-48e37b1265d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.050135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d557d-55df-4360-b471-48e37b1265d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.050397 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d557d-55df-4360-b471-48e37b1265d0 admin admin] PATH_INFO: `/v3/projects/9f822a7ca37e411f8a9edf1c4203ffbc/users/b61cc56d96cf4e15836a3389517526d3/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.065408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d557d-55df-4360-b471-48e37b1265d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f822a7ca37e411f8a9edf1c4203ffbc, user_id=b61cc56d96cf4e15836a3389517526d3, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.068079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d557d-55df-4360-b471-48e37b1265d0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.083237 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1169/2338] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:06 2022] PUT /identity/v3/projects/9f822a7ca37e411f8a9edf1c4203ffbc/users/b61cc56d96cf4e15836a3389517526d3/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:06.087078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.097088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc303841-818c-4532-9214-1a05568107b3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.175924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.176206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.176468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.176651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.177367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.177864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.178271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.179060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.180824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.186926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:06.195461 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1170/2339] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:01:06.196916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.197070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:06.197992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:06.207436 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf50cb-6ce6-4b36-bbd4-a8f4eacc065c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.213636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be0021-7945-49c7-b38b-454560c378e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.214352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be0021-7945-49c7-b38b-454560c378e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.214726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be0021-7945-49c7-b38b-454560c378e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.215099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be0021-7945-49c7-b38b-454560c378e9 admin admin] PATH_INFO: `/v3/projects/9f822a7ca37e411f8a9edf1c4203ffbc/users/b61cc56d96cf4e15836a3389517526d3/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.217131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be0021-7945-49c7-b38b-454560c378e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f822a7ca37e411f8a9edf1c4203ffbc, user_id=b61cc56d96cf4e15836a3389517526d3, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.220065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96be0021-7945-49c7-b38b-454560c378e9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.232253 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1171/2340] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:06 2022] PUT /identity/v3/projects/9f822a7ca37e411f8a9edf1c4203ffbc/users/b61cc56d96cf4e15836a3389517526d3/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:06.243794 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.244075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.244353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.263800 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1170/2341] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:01:06.276172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] MFA Rules not processed for user `b61cc56d96cf4e15836a3389517526d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:06.277654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:06.278676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.280268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:06.346802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:06.350850 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1172/2342] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:01:06.362075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.363529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:06.373212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7aa230-5ac1-4481-aa1f-4a4f97a9a862 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.426285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa3f61-ee10-432f-b55c-0e15d0adf382 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.431595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.432401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.432984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.433261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.438825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37dab24-400d-40ff-ac0d-75a2075e89ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.439529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dab24-400d-40ff-ac0d-75a2075e89ad service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.439877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dab24-400d-40ff-ac0d-75a2075e89ad service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.440213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37dab24-400d-40ff-ac0d-75a2075e89ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.443371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.444880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.455730 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1173/2343] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:06 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:01:06.534414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37dab24-400d-40ff-ac0d-75a2075e89ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.536547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37dab24-400d-40ff-ac0d-75a2075e89ad service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.554869 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1171/2344] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:01:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 01:01:06.658547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:06.790626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0423f35-288b-4929-8d3f-619040917df3 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:06.804757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:06.805582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.806094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.806403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] PATH_INFO: `/v3/users/c838d7c6c7fe4bdb82581cf592c30bfa` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:06.808984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c838d7c6c7fe4bdb82581cf592c30bfa)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:06.818386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:06.839934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:06.914940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c838d7c6c7fe4bdb82581cf592c30bfa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:06.926680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c838d7c6c7fe4bdb82581cf592c30bfa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:06.939299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c838d7c6c7fe4bdb82581cf592c30bfa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:06.942003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c838d7c6c7fe4bdb82581cf592c30bfa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:06.960106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c838d7c6c7fe4bdb82581cf592c30bfa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:06.971870 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1174/2345] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:06 2022] DELETE /identity/v3/users/c838d7c6c7fe4bdb82581cf592c30bfa => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:06.977703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:06.977895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:06.978100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:07.006635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:07.008502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:07.101045 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1172/2346] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:07.111002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:07.113634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:07.213737 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa733f0-9486-4399-adb6-52c96b3a800c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:07.219441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:07.220600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:07.220964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:07.221299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:07.240493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:07.242433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:07.260740 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1175/2347] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:07.748241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:07.749972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:07.764108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c2cc4-12ff-487c-8a0d-2a31e2d0f646 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:07.770033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:07.770814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:07.771258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:07.771640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] PATH_INFO: `/v3/projects/13d72aac78d5408991fa978fee19502e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:07.773680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d72aac78d5408991fa978fee19502e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:07.775768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:07.826643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:07.849514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d72aac78d5408991fa978fee19502e', 'request_id': 'req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:07.864107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d72aac78d5408991fa978fee19502e', 'request_id': 'req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:07.873389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d72aac78d5408991fa978fee19502e', 'request_id': 'req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:07.892789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] The token cache is being invalidate because project 13d72aac78d5408991fa978fee19502e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:07.893129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:07.893791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:07.895638 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1173/2348] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:07 2022] DELETE /identity/v3/projects/13d72aac78d5408991fa978fee19502e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:07.899596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:07.909952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b4097-b6bf-4431-b629-df10b46d7fe3 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:08.008786 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:08.009424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.009700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.010026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] PATH_INFO: `/v3/users/b61cc56d96cf4e15836a3389517526d3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.011999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b61cc56d96cf4e15836a3389517526d3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:08.014365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:08.077196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:08.084616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b61cc56d96cf4e15836a3389517526d3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.111839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b61cc56d96cf4e15836a3389517526d3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.133104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b61cc56d96cf4e15836a3389517526d3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.134207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b61cc56d96cf4e15836a3389517526d3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.151977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-733a1b72-bf59-4032-80ae-3411fca219cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b61cc56d96cf4e15836a3389517526d3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.159495 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1176/2349] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:07 2022] DELETE /identity/v3/users/b61cc56d96cf4e15836a3389517526d3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:01:08.472982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:08.474430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:08.488369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7e243-9148-4bbd-8edf-cdd3ec3d6da3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:08.571240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:08.572057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.572405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.572781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] PATH_INFO: `/v3/projects/9f822a7ca37e411f8a9edf1c4203ffbc` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.574040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f822a7ca37e411f8a9edf1c4203ffbc)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:08.576090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:08.627376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f822a7ca37e411f8a9edf1c4203ffbc', 'request_id': 'req-32198be5-dec0-4e2c-a631-ce0e0f72f74a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.636693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f822a7ca37e411f8a9edf1c4203ffbc', 'request_id': 'req-32198be5-dec0-4e2c-a631-ce0e0f72f74a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.642344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f822a7ca37e411f8a9edf1c4203ffbc', 'request_id': 'req-32198be5-dec0-4e2c-a631-ce0e0f72f74a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.654068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] The token cache is being invalidate because project 9f822a7ca37e411f8a9edf1c4203ffbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:08.654350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.654999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32198be5-dec0-4e2c-a631-ce0e0f72f74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:08.656801 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1174/2350] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:08 2022] DELETE /identity/v3/projects/9f822a7ca37e411f8a9edf1c4203ffbc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:08.661594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.661982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.662321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.684125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:08.685461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:08.753186 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1177/2351] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:01:08.761052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.761613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.761726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.789674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:08.789674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:08.795895 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1175/2352] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:08.800361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:08.810443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f895c0-a6c9-45fd-9b7a-5c16f8a8db7d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:08.815724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:08.816323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.816591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.816832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.817904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:08.818229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:08.819906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:08.831747 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1178/2353] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:01:08.836939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:08.847099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e4327-89f7-4a12-a43d-930c41fdd865 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:08.852611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:08.853321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.853644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.853979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.854747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:08.856786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:08.887728 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1176/2354] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:08 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:08.892180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:08.902243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39dc28-75a0-40c0-a589-499a5ef7190e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:08.967089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:08.967734 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:08.968008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:08.968272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:08.968906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:08.970840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:08.993774 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1179/2355] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:08 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:08.998260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.010992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fcedc1-78b9-4ee4-9d10-3a6fc4ca08b5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.016535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.017155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.017434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.017693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.018340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.020054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.034259 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1177/2356] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:09.038426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.049104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62a0d1-5199-44a5-9cbd-881c9483c1bb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.053548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9786a04-38c9-4b43-a7f2-7d02f5097b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.054136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9786a04-38c9-4b43-a7f2-7d02f5097b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.054407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9786a04-38c9-4b43-a7f2-7d02f5097b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.054672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9786a04-38c9-4b43-a7f2-7d02f5097b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.055355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9786a04-38c9-4b43-a7f2-7d02f5097b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.057045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9786a04-38c9-4b43-a7f2-7d02f5097b43 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.067955 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1180/2357] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:09.072471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.082856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc0676-14c7-48f9-a0d5-586197a261fd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.087592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.088287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.088604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.088876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] PATH_INFO: `/v3/projects/c852a2679de04a49a687799ee5e1aaa1/users/64be29eb3017465f879f01001feb94c9/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.092206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:09.106054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c852a2679de04a49a687799ee5e1aaa1, user_id=64be29eb3017465f879f01001feb94c9, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.109794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.124634 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1178/2358] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:09 2022] PUT /identity/v3/projects/c852a2679de04a49a687799ee5e1aaa1/users/64be29eb3017465f879f01001feb94c9/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:09.140748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.141959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.142413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.179329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] MFA Rules not processed for user `64be29eb3017465f879f01001feb94c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:09.180851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:09.191885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:09.271454 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1181/2359] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:09.275833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.358825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63ec04-bac0-4b94-9b99-45205805b808 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.364312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.365161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.365633 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.365970 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.366682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.368703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.417385 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1179/2360] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:09 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:09.421460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.433458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8f836-a4bc-4fc9-b06d-dd3c628b1aa3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.533160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.533778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.534051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.534340 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.534997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.537009 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.560752 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1182/2361] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:09 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:09.564735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.576167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82238adc-54bf-480e-ac80-076486bbd7dc admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.581760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.582389 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.582663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.583073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.583920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.585554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.597679 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1180/2362] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:09.604058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.615000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619bf32-712f-4330-b4c7-5a97bf09a00f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.620385 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.621078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.621335 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.621604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] PATH_INFO: `/v3/projects/162b5644c5ec4ef39f2606aa80bc923f/users/cbf94a027c72456aadc7bb7374375dd5/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.640470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162b5644c5ec4ef39f2606aa80bc923f, user_id=cbf94a027c72456aadc7bb7374375dd5, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.643450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.660925 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1183/2363] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:09 2022] PUT /identity/v3/projects/162b5644c5ec4ef39f2606aa80bc923f/users/cbf94a027c72456aadc7bb7374375dd5/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:09.665441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.676180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fea85-86ed-4b44-b18f-580a35832001 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.739142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c5356-2cf7-49f3-8b25-724e789411d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.739765 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c5356-2cf7-49f3-8b25-724e789411d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.740036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c5356-2cf7-49f3-8b25-724e789411d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.740300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c5356-2cf7-49f3-8b25-724e789411d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.740938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c5356-2cf7-49f3-8b25-724e789411d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.742521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c5356-2cf7-49f3-8b25-724e789411d6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.753003 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1181/2364] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:09.760058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.772649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1557930-6094-431a-828d-1f3ebccc71a3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:09.778586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:09.779536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.779936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.780310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] PATH_INFO: `/v3/projects/162b5644c5ec4ef39f2606aa80bc923f/users/cbf94a027c72456aadc7bb7374375dd5/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.782717 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=162b5644c5ec4ef39f2606aa80bc923f, user_id=cbf94a027c72456aadc7bb7374375dd5, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:09.786123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:09.799596 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1184/2365] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:09 2022] PUT /identity/v3/projects/162b5644c5ec4ef39f2606aa80bc923f/users/cbf94a027c72456aadc7bb7374375dd5/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:09.810036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:09.810814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:09.810959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:09.841308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] MFA Rules not processed for user `cbf94a027c72456aadc7bb7374375dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:09.842530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:09.886888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:09.927608 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1182/2366] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:09.935086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:09.936947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:10.134378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930feab-9b1f-4392-a265-50d6dc8cdaf5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:10.138448 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:10.139231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:10.144334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:10.144334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:10.158579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:10.160144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:10.171560 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1185/2367] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:10.258481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:10.268682 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2c9eb-8b8d-45b6-9dd6-b6e2170d42c1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:10.274298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:10.274980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:10.275409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:10.275689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:10.333931 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:10.335445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:10.347515 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1183/2368] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:16.215489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:16.217136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:16.228196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:16.280260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3cc04-cdf1-41ff-aaff-ed31af25250f service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:16.284596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:16.285210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:16.285486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:16.285778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:16.287723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:16.300287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:16.302426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a03be92-e8fb-4483-9a2c-54d816197055 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:16.314932 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1186/2369] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:17.811446 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:17.879612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45f3a6-697c-417f-b47c-6e2cd4600889 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:17.884090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:17.884722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:17.884989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:17.885294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] PATH_INFO: `/v3/users/64be29eb3017465f879f01001feb94c9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:17.886399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64be29eb3017465f879f01001feb94c9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:17.888453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:17.939548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64be29eb3017465f879f01001feb94c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:17.947601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64be29eb3017465f879f01001feb94c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:17.959448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64be29eb3017465f879f01001feb94c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:17.960436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64be29eb3017465f879f01001feb94c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:17.967971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64be29eb3017465f879f01001feb94c9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:17.969431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:17.975470 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1184/2370] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:17 2022] DELETE /identity/v3/users/64be29eb3017465f879f01001feb94c9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:17.980141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:17.980648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:17.980755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:18.000170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:18.001411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:18.070891 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1187/2371] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:18.080932 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:18.082449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:18.144463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8813667-5002-4924-8338-500d8247e517 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:18.149816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:18.150984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:18.151352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:18.151622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:18.162579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:18.164609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:18.174504 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1185/2372] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:18.450902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:18.452887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:18.468475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c710308-75a2-43c4-bd46-8568e481cc53 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:18.481054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:18.481726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:18.482053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:18.482326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] PATH_INFO: `/v3/projects/c852a2679de04a49a687799ee5e1aaa1` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:18.483518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c852a2679de04a49a687799ee5e1aaa1)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:18.485573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:18.538846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c852a2679de04a49a687799ee5e1aaa1', 'request_id': 'req-5f762fbd-f13c-4287-a39c-b0b0b133e45c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.548815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c852a2679de04a49a687799ee5e1aaa1', 'request_id': 'req-5f762fbd-f13c-4287-a39c-b0b0b133e45c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.558413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c852a2679de04a49a687799ee5e1aaa1', 'request_id': 'req-5f762fbd-f13c-4287-a39c-b0b0b133e45c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.571767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] The token cache is being invalidate because project c852a2679de04a49a687799ee5e1aaa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:18.572137 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.573485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.578200 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1188/2373] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:18 2022] DELETE /identity/v3/projects/c852a2679de04a49a687799ee5e1aaa1 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:18.579386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:18.589155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c67b5-8642-4014-b8b0-d4fe6417b06c service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:18.650887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:18.651583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:18.651909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:18.652182 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] PATH_INFO: `/v3/users/cbf94a027c72456aadc7bb7374375dd5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:18.653346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf94a027c72456aadc7bb7374375dd5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:18.655292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:18.701912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf94a027c72456aadc7bb7374375dd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.709557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf94a027c72456aadc7bb7374375dd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.721608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf94a027c72456aadc7bb7374375dd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.722249 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf94a027c72456aadc7bb7374375dd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.729048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f418b7b0-c997-47ad-a2aa-016152b399e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf94a027c72456aadc7bb7374375dd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:18.734895 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1186/2374] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:18 2022] DELETE /identity/v3/users/cbf94a027c72456aadc7bb7374375dd5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:19.028627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.033807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:19.050652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f762fbd-f13c-4287-a39c-b0b0b133e45c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.084742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:19.123407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.124085 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.124364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.124634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] PATH_INFO: `/v3/projects/162b5644c5ec4ef39f2606aa80bc923f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.125807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=162b5644c5ec4ef39f2606aa80bc923f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.131059 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.181761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '162b5644c5ec4ef39f2606aa80bc923f', 'request_id': 'req-40850705-1d54-42b8-b27a-9fecdcf306bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:19.189800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '162b5644c5ec4ef39f2606aa80bc923f', 'request_id': 'req-40850705-1d54-42b8-b27a-9fecdcf306bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:19.196590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '162b5644c5ec4ef39f2606aa80bc923f', 'request_id': 'req-40850705-1d54-42b8-b27a-9fecdcf306bf'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:19.208041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] The token cache is being invalidate because project 162b5644c5ec4ef39f2606aa80bc923f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:19.208321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:19.209054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-40850705-1d54-42b8-b27a-9fecdcf306bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:19.210571 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1189/2375] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:19 2022] DELETE /identity/v3/projects/162b5644c5ec4ef39f2606aa80bc923f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:19.215597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.216083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.216285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.236461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:19.237646 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:19.251388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:19.303040 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1187/2376] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:19.310856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.311140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.311449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.330681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:19.331907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:19.340350 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1190/2377] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:19.345168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.356824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79f428-8e51-4330-8834-f2cb8fa96387 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.362944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.363584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.363860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.364123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.364783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:19.365111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.366705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48029e3-f522-4502-b593-8fb12d58f6a5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.383561 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1188/2378] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:01:19.384596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.394354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4732167-5d80-4730-8fbb-3e1134402158 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.399052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.399691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.399965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.400298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.400863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.402788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.411869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.412155 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.412429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.432951 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1191/2379] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:19.434285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:19.435618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:19.437128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.494136 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1189/2380] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:01:19.503822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02ba06-f52f-4ea8-82d9-bc69529c49ef admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.507347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.508732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.508865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:19.509337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.509603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.509864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.510516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.512421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.537907 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1192/2381] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:19.539476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.549122 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699968f-be0c-4c78-a56d-2e272d6566f5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.554308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.554920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.555212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.555496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.556133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.557798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.571042 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1193/2382] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:19.572496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.582240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fb1e7-bd0d-4f9a-b82e-37e95a2cb214 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.584733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae9ea40-ad3f-406c-8e56-aeedf2adab1a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.587538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.588133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.588437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.588793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.589432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.590306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeea3b0-2a09-48bd-bc89-05ca4ec191a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.590917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea3b0-2a09-48bd-bc89-05ca4ec191a1 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.591029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.591220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea3b0-2a09-48bd-bc89-05ca4ec191a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.591493 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea3b0-2a09-48bd-bc89-05ca4ec191a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.600036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeea3b0-2a09-48bd-bc89-05ca4ec191a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.601617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeea3b0-2a09-48bd-bc89-05ca4ec191a1 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.601947 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1194/2383] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:01:19.606055 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:19.610051 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1190/2384] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:01:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 01:01:19.611294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:19.617183 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c98f9e4-8178-465c-a287-d83be21305c2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:19.623577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:19.624410 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.624509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.624797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] PATH_INFO: `/v3/projects/ce2a405fb0e44c8785d2598accee3e2f/users/feccff0ae9ad494ea82abd8a583131eb/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.645511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2a405fb0e44c8785d2598accee3e2f, user_id=feccff0ae9ad494ea82abd8a583131eb, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:19.648503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:19.686612 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1195/2385] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:19 2022] PUT /identity/v3/projects/ce2a405fb0e44c8785d2598accee3e2f/users/feccff0ae9ad494ea82abd8a583131eb/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:19.698739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:19.702352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:19.702644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:19.769966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] MFA Rules not processed for user `feccff0ae9ad494ea82abd8a583131eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:19.769966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:19.846052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:19.906572 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1191/2386] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:19.908170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.030499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985ca26-a159-4f60-ab44-4272f0df0d9f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.047251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.047251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.047251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.047251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.047772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.047772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.087274 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1196/2387] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:19 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:20.091014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.102936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91f780-b743-4249-a37b-a38e2bf2593e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.212230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.212939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.213299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.213943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.214657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.221575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.326306 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1192/2388] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:20 2022] POST /identity/v3/users => generated 291 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:20.327763 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.338692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ff71-fc55-4f53-9adc-32f06352f88c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.343520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.344149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.344402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.344704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.345329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.346943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.367279 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1197/2389] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:20.370427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.382619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee648577-185e-4fcf-94b1-48b23fc94057 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.387221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.388495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.388896 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.389497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] PATH_INFO: `/v3/projects/ce0ecd7881bc4d3f84557e24a32314aa/users/7bbacb85ea234d598f0d73ca9df33fab/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.408774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0ecd7881bc4d3f84557e24a32314aa, user_id=7bbacb85ea234d598f0d73ca9df33fab, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.411355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.427507 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1193/2390] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:20 2022] PUT /identity/v3/projects/ce0ecd7881bc4d3f84557e24a32314aa/users/7bbacb85ea234d598f0d73ca9df33fab/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:20.433599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.444563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f72474b-1ef9-4382-ab59-8fe833224cd0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.510784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4799821-64ed-49c9-a249-210deac894a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.511470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799821-64ed-49c9-a249-210deac894a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.511742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799821-64ed-49c9-a249-210deac894a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.512011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4799821-64ed-49c9-a249-210deac894a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.512668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4799821-64ed-49c9-a249-210deac894a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.514314 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4799821-64ed-49c9-a249-210deac894a6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.531346 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1198/2391] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:20.532301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.557253 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa31630-1fd6-4b95-8fb4-cafcb4184bbe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.562284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.563039 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.563444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.563777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] PATH_INFO: `/v3/projects/ce0ecd7881bc4d3f84557e24a32314aa/users/7bbacb85ea234d598f0d73ca9df33fab/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.565813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce0ecd7881bc4d3f84557e24a32314aa, user_id=7bbacb85ea234d598f0d73ca9df33fab, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.568747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.582940 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1194/2392] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:20 2022] PUT /identity/v3/projects/ce0ecd7881bc4d3f84557e24a32314aa/users/7bbacb85ea234d598f0d73ca9df33fab/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:20.593396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.593686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.593995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.632354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] MFA Rules not processed for user `7bbacb85ea234d598f0d73ca9df33fab`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:20.633853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:20.679283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:20.714664 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1199/2393] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:20.727861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.729678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:20.799030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c98118-7000-4de6-a92e-d4d60be98f45 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.804737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.806180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.806450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.806714 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.817463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.819158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.831509 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1195/2394] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:20.887882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:20.889319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:20.904983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06da544-d03f-41c5-8f6e-144847a21f6f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:20.908720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:20.909336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:20.909607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:20.909872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:20.980395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:20.981862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:20.992922 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1200/2395] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:21.331316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:21.333005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:21.383724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:21.457936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02137-87bb-431d-aa50-631f39f5003f service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:21.465288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:21.466058 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:21.466306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:21.466568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:21.468880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:21.536576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:21.539220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935a4c3-6a80-4d9d-8a82-817fbf1c2147 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:21.555677 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1196/2396] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:25.052025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:25.146092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790ee18-08bc-404f-b1bf-d9bd2296cf9a service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:25.153192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:25.159392 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:25.162001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:25.162397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] PATH_INFO: `/v3/users/feccff0ae9ad494ea82abd8a583131eb` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:25.164974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feccff0ae9ad494ea82abd8a583131eb)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:25.166686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:25.270605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feccff0ae9ad494ea82abd8a583131eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:25.302416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feccff0ae9ad494ea82abd8a583131eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:25.311349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feccff0ae9ad494ea82abd8a583131eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:25.312130 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feccff0ae9ad494ea82abd8a583131eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:25.327426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:25.333170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feccff0ae9ad494ea82abd8a583131eb'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:25.341630 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1201/2397] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:25 2022] DELETE /identity/v3/users/feccff0ae9ad494ea82abd8a583131eb => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:25.346545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:25.347741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:25.348148 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:25.371936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:25.373285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:25.445545 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1197/2398] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:25.455579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:25.457389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:25.534323 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a614e9-fede-481e-a237-c082f7d6e701 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:25.539045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:25.539789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:25.540157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:25.540517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:25.552258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:25.554019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:25.577550 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1202/2399] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:25.976049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:25.977521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:25.995483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51e709-b503-4cfb-a2c9-0e6a93043dde None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:26.008418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:26.009340 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:26.009816 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:26.010461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] PATH_INFO: `/v3/projects/ce2a405fb0e44c8785d2598accee3e2f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:26.013734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2a405fb0e44c8785d2598accee3e2f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:26.017105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:26.137732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2a405fb0e44c8785d2598accee3e2f', 'request_id': 'req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.151356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2a405fb0e44c8785d2598accee3e2f', 'request_id': 'req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.164718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:26.167740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2a405fb0e44c8785d2598accee3e2f', 'request_id': 'req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.190793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] The token cache is being invalidate because project ce2a405fb0e44c8785d2598accee3e2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:26.191082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.191813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.193561 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1198/2400] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:25 2022] DELETE /identity/v3/projects/ce2a405fb0e44c8785d2598accee3e2f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:26.197560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:26.218500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e0a9-e96a-4963-a897-0e3e7fe12851 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:26.433691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:26.434388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:26.434750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:26.434940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] PATH_INFO: `/v3/users/7bbacb85ea234d598f0d73ca9df33fab` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:26.436582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bbacb85ea234d598f0d73ca9df33fab)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:26.438832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:26.542371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bbacb85ea234d598f0d73ca9df33fab'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.573989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bbacb85ea234d598f0d73ca9df33fab'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.631479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bbacb85ea234d598f0d73ca9df33fab'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.631479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bbacb85ea234d598f0d73ca9df33fab'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.685639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e7c6427-185c-40a6-b66e-0a80a8566053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bbacb85ea234d598f0d73ca9df33fab'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:26.697377 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1203/2401] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:26 2022] DELETE /identity/v3/users/7bbacb85ea234d598f0d73ca9df33fab => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:01:27.090455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.092067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:27.139297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3afbf7-921d-4ed9-b939-4e8afcd2429c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.217545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.218200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.218475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.218749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] PATH_INFO: `/v3/projects/ce0ecd7881bc4d3f84557e24a32314aa` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.219898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce0ecd7881bc4d3f84557e24a32314aa)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.221811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.281243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce0ecd7881bc4d3f84557e24a32314aa', 'request_id': 'req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:27.290760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce0ecd7881bc4d3f84557e24a32314aa', 'request_id': 'req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:27.300519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce0ecd7881bc4d3f84557e24a32314aa', 'request_id': 'req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:27.318730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] The token cache is being invalidate because project ce0ecd7881bc4d3f84557e24a32314aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:27.319158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:27.319947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-36e0571b-cf16-4dd1-9dd5-3d27595ce31e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:27.321751 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1199/2402] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:27 2022] DELETE /identity/v3/projects/ce0ecd7881bc4d3f84557e24a32314aa => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:27.326882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.327443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.327649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.349367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:27.350765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:27.361650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:27.426062 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1204/2403] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:27.434479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.434711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.434966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.456711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:27.458025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:27.466771 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1200/2404] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:27.471203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.481867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7af3fc-998c-4a75-beb8-bfb022a7f811 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.487614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.488484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.488913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.491816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.491816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:27.491816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.492127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.509576 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1205/2405] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:01:27.510438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.520637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdd074a-8346-43f9-88a4-2a09f9327273 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.526278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.526893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.527221 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.527462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.528113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.530545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.570715 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1201/2406] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:27 2022] POST /identity/v3/projects => generated 399 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:27.572299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.586915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86e801-0b98-443d-8751-51ecc482146f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.669366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.670170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.670547 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.670913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.671696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.673985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.707365 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1206/2407] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:27 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:27.710848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.726901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea32746-7399-40f4-90a3-0f8f670af957 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.736575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.737356 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.737729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.738098 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.738856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.740864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.758069 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1202/2408] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:27.759940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.774947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828ed63-d54b-492a-8da0-23e97a60a69b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.781470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956dd2b-d8a8-4c4f-a5a2-b28432017fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.782228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956dd2b-d8a8-4c4f-a5a2-b28432017fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.782506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956dd2b-d8a8-4c4f-a5a2-b28432017fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.782785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956dd2b-d8a8-4c4f-a5a2-b28432017fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.783481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956dd2b-d8a8-4c4f-a5a2-b28432017fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.785138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956dd2b-d8a8-4c4f-a5a2-b28432017fcb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.800949 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1207/2409] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:27.805973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:27.821192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda915d7-b37b-4296-8a24-9f16dd4bdca6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:27.845519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:27.846510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.847492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.848051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] PATH_INFO: `/v3/projects/614dcedd45b84e1ebf3a369cf04e0a54/users/4ac6259a215f4f0db11217a803354344/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:27.898880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614dcedd45b84e1ebf3a369cf04e0a54, user_id=4ac6259a215f4f0db11217a803354344, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:27.898880 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:27.947548 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1203/2410] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:27 2022] PUT /identity/v3/projects/614dcedd45b84e1ebf3a369cf04e0a54/users/4ac6259a215f4f0db11217a803354344/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:27.954605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:27.955213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:27.955651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.002115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] MFA Rules not processed for user `4ac6259a215f4f0db11217a803354344`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:28.002515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:28.120543 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1208/2411] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:28.125282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:28.152057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:28.240382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211b8fa-8643-49a9-86b0-a66a35c48e33 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:28.245952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:28.246566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.246961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.247272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.247922 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:28.250495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:28.291838 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1204/2412] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:28 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:28.296331 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:28.309121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feff3f-daac-419d-8faa-4d68ebc95417 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:28.318791 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:28.447423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:28.448327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.449211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.449504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.450249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:28.452657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:28.488013 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1209/2413] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:28 2022] POST /identity/v3/users => generated 307 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:28.491943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:28.507859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb806d5-382f-4b1e-bf82-027bd9a2b6e5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:28.520860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:28.521719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.521995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.522357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.523011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:28.524707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:28.538991 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1205/2414] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:28.548390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:28.559198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc9d81-7def-44a3-a716-60752136155c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:28.564062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:28.564732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.565011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.565279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] PATH_INFO: `/v3/projects/139c002c9f2645d2b569949ccb56bdd9/users/a197ca800a3644afaaeb5ff1176e2d86/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.590500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=139c002c9f2645d2b569949ccb56bdd9, user_id=a197ca800a3644afaaeb5ff1176e2d86, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:28.593435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:28.618579 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1210/2415] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:28 2022] PUT /identity/v3/projects/139c002c9f2645d2b569949ccb56bdd9/users/a197ca800a3644afaaeb5ff1176e2d86/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:28.622964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:28.638001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea420a5c-5c89-4fdd-8bec-7a3d04d38f18 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:28.750189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed2214-bc47-4e07-a29d-1548dcf98db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:28.751060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed2214-bc47-4e07-a29d-1548dcf98db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.751477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed2214-bc47-4e07-a29d-1548dcf98db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.751843 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed2214-bc47-4e07-a29d-1548dcf98db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.752593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed2214-bc47-4e07-a29d-1548dcf98db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:28.754290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed2214-bc47-4e07-a29d-1548dcf98db9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:28.767150 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1206/2416] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:28 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:28.771058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:28.788181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e155e-2b0f-464c-9eec-dfb4e92642a6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:28.806460 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:28.807223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.807560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.807948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] PATH_INFO: `/v3/projects/139c002c9f2645d2b569949ccb56bdd9/users/a197ca800a3644afaaeb5ff1176e2d86/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.809910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] RBAC: Authorizing `identity:create_grant(project_id=139c002c9f2645d2b569949ccb56bdd9, user_id=a197ca800a3644afaaeb5ff1176e2d86, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:28.812822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:28.835275 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1211/2417] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:28 2022] PUT /identity/v3/projects/139c002c9f2645d2b569949ccb56bdd9/users/a197ca800a3644afaaeb5ff1176e2d86/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:01:28.837961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:28.838306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:28.838741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:28.873910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] MFA Rules not processed for user `a197ca800a3644afaaeb5ff1176e2d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:28.876454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:29.004114 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1207/2418] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:28 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:29.015363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:29.018733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:29.123488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e893e9b-6adc-40e2-9faa-b721d946367b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:29.128068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:29.128712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:29.129719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:29.130056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:29.149329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:29.150955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:29.167231 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1212/2419] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:29 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:29.268158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:29.279251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6c209-353f-41bb-892a-f0ceaf85bfec None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:29.283005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:29.283826 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:29.284202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:29.284569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:29.294106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:29.376770 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:29.378665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:29.389272 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1208/2420] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:32.375789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:32.377346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:32.440223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2044c-c856-4dda-a0f4-ad05f563972a service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:32.444261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:32.444878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:32.445176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:32.445499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:32.447157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:32.465950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:32.525856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:32.527479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:32.538970 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1213/2421] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:33.612574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:33.614445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:33.706291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b465536-d459-400f-8d8a-16a462bf9dfb service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:33.715578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:33.716686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:33.717325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:33.717325 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:33.719109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:33.724393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:33.790490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:33.792740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3447f04-d2f0-4417-b69a-bb39d023a9ed service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:33.797348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c9c4e-0129-4cdb-9337-d0999be569b3 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:33.802186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:33.803008 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1209/2422] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:33.803152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:33.803279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:33.803622 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] PATH_INFO: `/v3/users/4ac6259a215f4f0db11217a803354344` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:33.804918 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac6259a215f4f0db11217a803354344)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:33.807043 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:33.866833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac6259a215f4f0db11217a803354344'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:33.877280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac6259a215f4f0db11217a803354344'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:33.885735 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac6259a215f4f0db11217a803354344'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:33.886534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac6259a215f4f0db11217a803354344'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:33.903678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac6259a215f4f0db11217a803354344'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:33.910957 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1214/2423] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:33 2022] DELETE /identity/v3/users/4ac6259a215f4f0db11217a803354344 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:33.916730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:33.917015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:33.917459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:33.939641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:33.940859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:33.982478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:34.005014 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1210/2424] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:34.015764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:34.017488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:34.067618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:34.094873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be02287-aa11-48b6-9de4-d58d5de28fc2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:34.107294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:34.107294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:34.107294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:34.107294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:34.133285 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:34.135122 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:34.150279 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1215/2425] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:34.333128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:34.334543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:34.458487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd518cf-48da-41f3-8e75-25372c136a22 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:34.469923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:34.471332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:34.472679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:34.472679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:34.476613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:34.502169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:34.503985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:34.514897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:34.517363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:34.522649 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1211/2426] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:34.538628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c445c4-12ba-4aa8-8dc5-bc8d6b3d4434 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:34.545115 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:34.545903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:34.546234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:34.546623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] PATH_INFO: `/v3/projects/614dcedd45b84e1ebf3a369cf04e0a54` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:34.547991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614dcedd45b84e1ebf3a369cf04e0a54)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:34.554007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:34.650850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614dcedd45b84e1ebf3a369cf04e0a54', 'request_id': 'req-8fbafa66-a00f-4aca-b861-a36547a68105'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.661888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614dcedd45b84e1ebf3a369cf04e0a54', 'request_id': 'req-8fbafa66-a00f-4aca-b861-a36547a68105'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.672063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614dcedd45b84e1ebf3a369cf04e0a54', 'request_id': 'req-8fbafa66-a00f-4aca-b861-a36547a68105'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.690638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] The token cache is being invalidate because project 614dcedd45b84e1ebf3a369cf04e0a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:34.690942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.692070 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.697473 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1216/2427] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:34 2022] DELETE /identity/v3/projects/614dcedd45b84e1ebf3a369cf04e0a54 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:34.698994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:34.714429 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f4e6d-9560-43c9-960d-1b44bf866630 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:34.818905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:34.819855 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:34.820234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:34.820659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] PATH_INFO: `/v3/users/a197ca800a3644afaaeb5ff1176e2d86` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:34.822526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a197ca800a3644afaaeb5ff1176e2d86)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:34.824815 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:34.932798 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a197ca800a3644afaaeb5ff1176e2d86'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.948392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a197ca800a3644afaaeb5ff1176e2d86'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.984556 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a197ca800a3644afaaeb5ff1176e2d86'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.986554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a197ca800a3644afaaeb5ff1176e2d86'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:34.991008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:35.018439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-80c6204e-d5ab-4ddb-9839-08254e08499a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a197ca800a3644afaaeb5ff1176e2d86'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:35.030706 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1212/2428] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:34 2022] DELETE /identity/v3/users/a197ca800a3644afaaeb5ff1176e2d86 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:35.472503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:35.474638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:35.494305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbafa66-a00f-4aca-b861-a36547a68105 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:35.614997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:35.616174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:35.616395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:35.616663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] PATH_INFO: `/v3/projects/139c002c9f2645d2b569949ccb56bdd9` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:35.617927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=139c002c9f2645d2b569949ccb56bdd9)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:35.620127 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:35.730612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '139c002c9f2645d2b569949ccb56bdd9', 'request_id': 'req-e47189fd-fbcb-43e4-bf83-932584da8ecc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:35.745022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '139c002c9f2645d2b569949ccb56bdd9', 'request_id': 'req-e47189fd-fbcb-43e4-bf83-932584da8ecc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:35.754473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '139c002c9f2645d2b569949ccb56bdd9', 'request_id': 'req-e47189fd-fbcb-43e4-bf83-932584da8ecc'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:35.756062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:35.779324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] The token cache is being invalidate because project 139c002c9f2645d2b569949ccb56bdd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:35.779457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:35.780416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e47189fd-fbcb-43e4-bf83-932584da8ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:35.786974 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1217/2429] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:35 2022] DELETE /identity/v3/projects/139c002c9f2645d2b569949ccb56bdd9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:35.789487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:35.790011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:35.791090 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:35.828218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:35.830507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:36.016143 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1213/2430] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:36.026567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.026971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.027807 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.086419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:36.089051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:36.115528 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1218/2431] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:36.119711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.157570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe07d4-e9fc-480f-a0d1-dc1c3b4a552a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.178534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.179371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.179639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.180088 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.180724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:36.181023 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.182620 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.200976 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1214/2432] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:01:36.205384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.218972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51739f0f-bbec-4f6f-b541-a2055c1dd730 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.226984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.227872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.228838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.229182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.230014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.232147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.269625 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1219/2433] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:36 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:36.274053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.284612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613eb6f-7d28-48a7-af87-946ab6f9afbd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.314754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.317249 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:36.367634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.368254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.368526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.368789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.369592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.371666 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.407241 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1215/2434] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:36 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:36.411411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.421212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf4c10-d05b-4809-bf33-6ea50680b48a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.427634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.428312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.428647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.428915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.429573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.431273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.446292 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1216/2435] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:36.447938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462282aa-fed6-4da4-b3d2-d8c44342bb1e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.448099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.454431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.455051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.455442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.458069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.458362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6e77ea-a9b6-4f82-bb74-724bcf05fdf9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.459934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:36.463813 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.464404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.464706 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.464973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.465937 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.468223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.474065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.478106 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2ddc3-b9ab-46a1-a29e-8cb36384b3c0 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.482066 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1217/2436] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:36.486188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.489931 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1220/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:01:36.497111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25fc3da-5669-4443-9129-5d8b181d9d36 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.502416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.503080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.503384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.503690 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] PATH_INFO: `/v3/projects/6b1f92f790a043bf9848695a5c517857/users/26e7fce78cc74e0aace9b0956762c126/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.527198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1f92f790a043bf9848695a5c517857, user_id=26e7fce78cc74e0aace9b0956762c126, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.531270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.549699 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1218/2438] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:36 2022] PUT /identity/v3/projects/6b1f92f790a043bf9848695a5c517857/users/26e7fce78cc74e0aace9b0956762c126/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:01:36.554781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.555068 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.555370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.578400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:36.579712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:36.651356 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1221/2439] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:36.680016 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:36.682852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:36.720354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:36.790700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519fa40-a56f-411b-808a-ad74172c64d9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:36.796515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df080f6e-389b-4297-8c0e-3dc38e74ca40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:36.797780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df080f6e-389b-4297-8c0e-3dc38e74ca40 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:36.798280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df080f6e-389b-4297-8c0e-3dc38e74ca40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:36.798977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df080f6e-389b-4297-8c0e-3dc38e74ca40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:36.811365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df080f6e-389b-4297-8c0e-3dc38e74ca40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:36.813021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df080f6e-389b-4297-8c0e-3dc38e74ca40 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:36.825361 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1219/2440] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:37.114991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:37.125755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1920545-4d0b-48c9-a73f-ce1d50c6d697 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:37.129956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:37.130578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.130849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.131113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:37.133438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:37.145284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:37.146874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b17cf-8f5a-4963-bac7-79402608ef15 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:37.158852 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1222/2441] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:37.359779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042efc92-a30b-4509-a9d3-91fb4106c007 None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.360344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042efc92-a30b-4509-a9d3-91fb4106c007 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.360463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042efc92-a30b-4509-a9d3-91fb4106c007 None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:37.362165 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1220/2442] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 01:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 01:01:37.370530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363981d-ecce-49d3-bf78-1330ad50544a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.370840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363981d-ecce-49d3-bf78-1330ad50544a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.371375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363981d-ecce-49d3-bf78-1330ad50544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:37.406706 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8363981d-ecce-49d3-bf78-1330ad50544a None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:37.408263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8363981d-ecce-49d3-bf78-1330ad50544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:37.499912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:37.590314 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1223/2443] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 01:01:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:37.606063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042efc92-a30b-4509-a9d3-91fb4106c007 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:37.608073 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-042efc92-a30b-4509-a9d3-91fb4106c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:37.729869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042efc92-a30b-4509-a9d3-91fb4106c007 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:37.733925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353e627-b04a-40c6-88f2-156548accfde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:37.734737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353e627-b04a-40c6-88f2-156548accfde service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.735206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353e627-b04a-40c6-88f2-156548accfde service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.735479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353e627-b04a-40c6-88f2-156548accfde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:37.750946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353e627-b04a-40c6-88f2-156548accfde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:37.752413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5353e627-b04a-40c6-88f2-156548accfde service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:37.768052 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1221/2444] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:37.852214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338acd94-db1c-455c-8bd7-fed4364afc08 None None] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.852877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338acd94-db1c-455c-8bd7-fed4364afc08 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.853212 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338acd94-db1c-455c-8bd7-fed4364afc08 None None] PATH_INFO: `/` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:37.855602 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1224/2445] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 01:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 11 01:01:37.861974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88d72-dd57-4be5-bb92-7729ddca766d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.863685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88d72-dd57-4be5-bb92-7729ddca766d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.864042 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed88d72-dd57-4be5-bb92-7729ddca766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:37.892407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fed88d72-dd57-4be5-bb92-7729ddca766d None None] MFA Rules not processed for user `341595ae0d9b4431b4417edda29f03ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:37.895537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fed88d72-dd57-4be5-bb92-7729ddca766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:37.911649 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1222/2446] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 01:01:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:37.932800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338acd94-db1c-455c-8bd7-fed4364afc08 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:37.936863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-338acd94-db1c-455c-8bd7-fed4364afc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:37.977835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338acd94-db1c-455c-8bd7-fed4364afc08 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:37.987128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e136625a-36d6-4f64-8abb-6f12c1bed7cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:37.988279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136625a-36d6-4f64-8abb-6f12c1bed7cf service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:37.989172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136625a-36d6-4f64-8abb-6f12c1bed7cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:37.989664 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e136625a-36d6-4f64-8abb-6f12c1bed7cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.001707 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136625a-36d6-4f64-8abb-6f12c1bed7cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:38.003966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e136625a-36d6-4f64-8abb-6f12c1bed7cf service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:38.019163 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1225/2447] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:38.182621 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aeecf-2390-4083-a448-0b81abc9260e None None] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.182908 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aeecf-2390-4083-a448-0b81abc9260e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.183162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9aeecf-2390-4083-a448-0b81abc9260e None None] PATH_INFO: `/` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.185015 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1223/2448] 10.222.0.23 () {54 vars in 921 bytes} [Sun Sep 11 01:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 11 01:01:38.191261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea1560-7d64-4feb-979f-3947f11d24df None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.191629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea1560-7d64-4feb-979f-3947f11d24df None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.191966 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea1560-7d64-4feb-979f-3947f11d24df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.231634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0cea1560-7d64-4feb-979f-3947f11d24df None None] MFA Rules not processed for user `c150143af9144547ab3dc5a5d318d45f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:38.233467 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0cea1560-7d64-4feb-979f-3947f11d24df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:38.255941 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1226/2449] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Sep 11 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:38.275846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aeecf-2390-4083-a448-0b81abc9260e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:38.294869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9aeecf-2390-4083-a448-0b81abc9260e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:38.300223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:38.300841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.301111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.301373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.322363 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:38.324196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:38.338789 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1224/2450] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:38.440022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.440374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.440779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.472798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] MFA Rules not processed for user `26e7fce78cc74e0aace9b0956762c126`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:38.474309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:38.552277 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1227/2451] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:38.556582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:38.576352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bd7b5-3817-44c6-a137-0d1d9a413ea7 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:38.588245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:38.588869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.589162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.590819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.591520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:38.593438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:38.681872 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1225/2452] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:38 2022] POST /identity/v3/projects => generated 371 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:38.684774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:38.703764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f4dd1-9f1f-43e5-896b-d909b6b1438e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:38.810013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:38.810926 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.811485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.811758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.812499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:38.814710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:38.846683 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1228/2453] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:38 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:38.850707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:38.861799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c9e66-71ed-4adb-824c-052cde000da8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:38.866411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:38.867014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.867306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.867578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.868228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:38.869807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:38.880960 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1226/2454] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:38.884817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:38.896388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45875cbf-e133-4f8d-a1d7-a1e23d024cf8 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:38.899820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:38.900479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:38.900753 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:38.901022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] PATH_INFO: `/v3/projects/0c6cf8012e2f454ab56fcd57289c4449/users/6861061d30fb4f13b389116d73653400/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:38.917673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6cf8012e2f454ab56fcd57289c4449, user_id=6861061d30fb4f13b389116d73653400, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:38.921344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:38.931424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:38.937358 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1229/2455] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:38 2022] PUT /identity/v3/projects/0c6cf8012e2f454ab56fcd57289c4449/users/6861061d30fb4f13b389116d73653400/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:01:38.941313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:38.951241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a439dd8-735b-4267-8a52-4971b0a94a2c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:39.013744 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b68bf-62f0-4281-921e-255f0e82c5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:39.014994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b68bf-62f0-4281-921e-255f0e82c5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:39.015308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b68bf-62f0-4281-921e-255f0e82c5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:39.015979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b68bf-62f0-4281-921e-255f0e82c5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:39.016644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b68bf-62f0-4281-921e-255f0e82c5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:39.018579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b68bf-62f0-4281-921e-255f0e82c5d4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:39.035699 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1227/2456] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:39.036694 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:39.055556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7541-eb6b-452d-8df3-3141f6d10383 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:39.061880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002a4a2-a8ab-478c-863e-7dd02c5fb96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:39.062406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002a4a2-a8ab-478c-863e-7dd02c5fb96e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:39.062681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002a4a2-a8ab-478c-863e-7dd02c5fb96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:39.062954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002a4a2-a8ab-478c-863e-7dd02c5fb96e admin admin] PATH_INFO: `/v3/projects/0c6cf8012e2f454ab56fcd57289c4449/users/6861061d30fb4f13b389116d73653400/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:39.064905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002a4a2-a8ab-478c-863e-7dd02c5fb96e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c6cf8012e2f454ab56fcd57289c4449, user_id=6861061d30fb4f13b389116d73653400, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:39.067557 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002a4a2-a8ab-478c-863e-7dd02c5fb96e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:39.082100 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1230/2457] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:39 2022] PUT /identity/v3/projects/0c6cf8012e2f454ab56fcd57289c4449/users/6861061d30fb4f13b389116d73653400/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:01:39.889956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:39.890515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:39.890613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:39.912694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:39.922572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] MFA Rules not processed for user `6861061d30fb4f13b389116d73653400`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:39.924990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:39.998215 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1228/2458] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:40.069198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622631-068f-4d2c-bbe5-2e61471e5505 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:40.069951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622631-068f-4d2c-bbe5-2e61471e5505 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:40.069951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b622631-068f-4d2c-bbe5-2e61471e5505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:40.090125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8b622631-068f-4d2c-bbe5-2e61471e5505 None None] MFA Rules not processed for user `26e7fce78cc74e0aace9b0956762c126`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:40.091442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8b622631-068f-4d2c-bbe5-2e61471e5505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:40.147802 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1231/2459] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:40.159876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:40.161865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:40.237362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bc025-55d4-445a-803d-4b513a04e907 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:40.243716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:40.244330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:40.244594 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:40.244861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:40.253844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:40.256413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:40.267861 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1229/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:45.587246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:45.588255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:45.588681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:45.618548 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:45.619984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:45.701750 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1232/2461] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 01:01:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:45.714099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:45.715805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:45.819610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae998f-4d90-4ca3-8aae-b1a33130b52a service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:45.827792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:45.831964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:45.832394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:45.832651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:45.851145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:45.853667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:45.863911 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1230/2462] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:01:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 01:01:46.381982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:46.401357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ab413-454e-4052-83d6-99a290a76e37 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:46.416601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:46.417237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:46.417509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:46.417990 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:46.424956 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:46.458605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:46.461159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:46.491073 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1233/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:51.390141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:51.407632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:51.462370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da546f-0906-4f64-ab37-7496b4459223 service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:51.466062 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:51.466668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:51.466942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:51.467220 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:51.468936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:51.548970 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:51.550462 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:51.578321 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1231/2464] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:55.918384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:55.920081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:55.934825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b123d82-2af9-4a40-b90c-04a4473a0852 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:55.940248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:55.940919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:55.941800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:55.941800 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] PATH_INFO: `/v3/users/081d365c7b15443aa76c7da2ca444a8e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:55.942758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081d365c7b15443aa76c7da2ca444a8e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:55.944872 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.000471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081d365c7b15443aa76c7da2ca444a8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.012680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081d365c7b15443aa76c7da2ca444a8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.027204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081d365c7b15443aa76c7da2ca444a8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.029839 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081d365c7b15443aa76c7da2ca444a8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.043939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:56.046084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a4f54cda-2aaf-4596-b78b-1cfb2a8b6033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081d365c7b15443aa76c7da2ca444a8e'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.055279 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1234/2465] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:55 2022] DELETE /identity/v3/users/081d365c7b15443aa76c7da2ca444a8e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:56.384536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:56.397353 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d238899-d2dd-4903-90bd-af0db5e8d9b9 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:56.466794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:56.467463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.467760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.468027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] PATH_INFO: `/v3/projects/4dec451848364c2787673580ec4c90c1` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.469146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dec451848364c2787673580ec4c90c1)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:56.471041 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.518794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dec451848364c2787673580ec4c90c1', 'request_id': 'req-e32282d8-2545-434f-94f0-23cfa444531a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.526750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dec451848364c2787673580ec4c90c1', 'request_id': 'req-e32282d8-2545-434f-94f0-23cfa444531a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.530740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:56.534052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dec451848364c2787673580ec4c90c1', 'request_id': 'req-e32282d8-2545-434f-94f0-23cfa444531a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.551361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] The token cache is being invalidate because project 4dec451848364c2787673580ec4c90c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:56.551679 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.552430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e32282d8-2545-434f-94f0-23cfa444531a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:56.557531 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1232/2466] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:56 2022] DELETE /identity/v3/projects/4dec451848364c2787673580ec4c90c1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:56.559330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.559610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.559880 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.582403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:56.583798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:56.667026 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1235/2467] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:56.675077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.675490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.675867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.700123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:56.701084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:56.712830 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1233/2468] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:56.717795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:56.729205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a4d47-8f39-4107-9112-3b83fdfbcfca None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:56.738915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:56.739655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.740148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.740479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.741308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:56.741738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:56.743809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.757095 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1236/2469] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:01:56.762028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:56.771960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb548f04-98e6-49c0-8238-7a830115ac3a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:56.777404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:56.778265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.778632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.778960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.779723 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:56.781705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.812505 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1234/2470] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:56 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:56.817810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:56.830231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d4697-5225-4914-aaee-2b2d9ca88ffa admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:56.893236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:56.893992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.894444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.894812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.895538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:56.897554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.920577 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1237/2471] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:56 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:56.924939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:56.935640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d522c94-4a95-4cf3-b6a4-9160a92071a9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:56.941069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:56.942035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.942035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.942240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.942889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:56.944525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.954842 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1235/2472] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:56.959063 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:56.969570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297b327-b23b-48d4-bf05-f6b1befc71c9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:56.974345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d35b3-520f-4b79-a5d3-18ebbaf1d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:56.974945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d35b3-520f-4b79-a5d3-18ebbaf1d4be admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:56.975262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d35b3-520f-4b79-a5d3-18ebbaf1d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:56.975523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d35b3-520f-4b79-a5d3-18ebbaf1d4be admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:56.976167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d35b3-520f-4b79-a5d3-18ebbaf1d4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:56.977758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d35b3-520f-4b79-a5d3-18ebbaf1d4be admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:56.988501 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1238/2473] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:56.992866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:57.003286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee71d2-9156-4509-857a-80e572132420 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:57.007998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:57.008692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:57.009259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:57.009663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] PATH_INFO: `/v3/projects/5fb9f54b53244122859ad3caf974596e/users/d078b719015d43ce9143ccb3e7870615/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:57.027742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb9f54b53244122859ad3caf974596e, user_id=d078b719015d43ce9143ccb3e7870615, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:57.030505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:57.059784 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1236/2474] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:56 2022] PUT /identity/v3/projects/5fb9f54b53244122859ad3caf974596e/users/d078b719015d43ce9143ccb3e7870615/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:57.061795 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:57.062079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:57.062350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:57.093397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] MFA Rules not processed for user `d078b719015d43ce9143ccb3e7870615`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:57.094780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:57.155089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:57.177044 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1239/2475] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:57.189035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:57.190848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:57.279254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73862495-7014-46a5-80a6-34f6b18dc7e1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:57.285218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4551274c-23dd-4d33-8bf4-4e7d9862dc86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:57.286100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551274c-23dd-4d33-8bf4-4e7d9862dc86 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:57.286444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551274c-23dd-4d33-8bf4-4e7d9862dc86 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:57.286710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4551274c-23dd-4d33-8bf4-4e7d9862dc86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:57.303495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551274c-23dd-4d33-8bf4-4e7d9862dc86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:57.306466 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4551274c-23dd-4d33-8bf4-4e7d9862dc86 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:57.319852 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1237/2476] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:57.630653 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:57.725992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfaafbf-024d-47b9-970a-e1b6d41d559c None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:57.730533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:57.731194 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:57.731475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:57.731742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] PATH_INFO: `/v3/users/d078b719015d43ce9143ccb3e7870615` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:57.732904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d078b719015d43ce9143ccb3e7870615)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:57.734820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:57.796731 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd078b719015d43ce9143ccb3e7870615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:57.807402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd078b719015d43ce9143ccb3e7870615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:57.818005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd078b719015d43ce9143ccb3e7870615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:57.818886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd078b719015d43ce9143ccb3e7870615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:57.840434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd078b719015d43ce9143ccb3e7870615'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:57.847750 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1240/2477] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:01:57 2022] DELETE /identity/v3/users/d078b719015d43ce9143ccb3e7870615 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:57.852229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:57.852791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:57.852989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:57.876862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:57.878213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:57.945172 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1238/2478] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:57.957177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:57.958750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:58.018517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2399dea6-9c9a-4f32-89b8-32f6f19f0b4c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.022453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8a9d7-845a-4137-b278-8cc502409594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.023065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8a9d7-845a-4137-b278-8cc502409594 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.023369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8a9d7-845a-4137-b278-8cc502409594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.023640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8a9d7-845a-4137-b278-8cc502409594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.033431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8a9d7-845a-4137-b278-8cc502409594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.034835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8a9d7-845a-4137-b278-8cc502409594 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.045482 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1241/2479] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:58.320255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.321495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:58.344965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f144e-2ce7-4cac-b59e-207b291216c3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.349848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.350577 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.351085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.351513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] PATH_INFO: `/v3/projects/5fb9f54b53244122859ad3caf974596e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.352773 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb9f54b53244122859ad3caf974596e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.354799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.363724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:58.407141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb9f54b53244122859ad3caf974596e', 'request_id': 'req-64dec755-0feb-4643-9ac4-8a5b10678ad9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:58.415370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb9f54b53244122859ad3caf974596e', 'request_id': 'req-64dec755-0feb-4643-9ac4-8a5b10678ad9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:58.421619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb9f54b53244122859ad3caf974596e', 'request_id': 'req-64dec755-0feb-4643-9ac4-8a5b10678ad9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:58.432887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] The token cache is being invalidate because project 5fb9f54b53244122859ad3caf974596e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:01:58.433167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:58.433905 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-64dec755-0feb-4643-9ac4-8a5b10678ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:01:58.435468 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1239/2480] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:01:58 2022] DELETE /identity/v3/projects/5fb9f54b53244122859ad3caf974596e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:58.440750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.441032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.441296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.461638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:58.462843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:58.471660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:58.524847 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1242/2481] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:58.537436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.538154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.538518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.565313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:58.566518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:58.574408 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1240/2482] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:58.578841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.592141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714b46d3-f06c-4c3a-8299-48e4403481fe None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.598150 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.598753 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.598998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.599327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.599979 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:01:58.600276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.601862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.614152 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1243/2483] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:01:58.618688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.628273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a771b8f-9de6-49ae-b6d8-6374ac7406ef None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.633199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.633788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.634056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.634322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.634961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.636921 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.670126 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1241/2484] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:58 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:58.674037 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.684822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f000bdb-3ec8-4152-bdcf-af550221e323 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.763760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.763760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.763760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.763760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.765855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.765855 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.789592 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1244/2485] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:58 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:58.793510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.805779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406988e-9a80-433f-a581-60f0b16351b2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.811351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.812095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.812426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.812739 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.813422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.815925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.827203 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1242/2486] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:58.829743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.843371 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e88367-4ff3-447a-82ca-01e83d558ce7 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.854216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558afb7-b999-4d7a-9d71-4151fbd2b3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.854629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558afb7-b999-4d7a-9d71-4151fbd2b3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.855028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558afb7-b999-4d7a-9d71-4151fbd2b3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.855665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558afb7-b999-4d7a-9d71-4151fbd2b3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.856632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558afb7-b999-4d7a-9d71-4151fbd2b3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.858693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0558afb7-b999-4d7a-9d71-4151fbd2b3b5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.874370 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1245/2487] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:58.878887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:58.890581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e249f8-b58d-4f35-b966-fe33cd5489ff admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:58.895454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:58.896112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.896382 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.896651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] PATH_INFO: `/v3/projects/b4224af1c6304bf2ba9307b711aa2302/users/1407fd80e2dd4516825c0d2e16016d88/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.915417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4224af1c6304bf2ba9307b711aa2302, user_id=1407fd80e2dd4516825c0d2e16016d88, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:58.918173 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:58.932479 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1243/2488] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:58 2022] PUT /identity/v3/projects/b4224af1c6304bf2ba9307b711aa2302/users/1407fd80e2dd4516825c0d2e16016d88/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:58.943848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:58.944121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:58.944387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:58.974796 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] MFA Rules not processed for user `1407fd80e2dd4516825c0d2e16016d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:58.976365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:59.046015 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1246/2489] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:01:59.050599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.116322 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5eb9a5-ce7e-47fb-991c-646e53b2d48f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.121451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.122140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.122481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.122812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.123539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.125505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.156497 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1244/2490] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:01:59 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:01:59.158376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.170191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc334c-7096-4da9-b559-1303e204efd3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.175162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:59.229518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.230132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.230444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.230741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.231393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.233441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.256322 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1247/2491] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:01:59 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:01:59.260634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.270955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb18b3-1b62-480b-9282-200a48d61283 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.275636 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.276229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.276497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.276760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.277492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.279087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.290421 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1245/2492] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:01:59.294935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.305629 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd8eab-9d23-45ce-8be3-7a94a03c5c29 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.311745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.312393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.312663 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.312928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] PATH_INFO: `/v3/projects/9481f0f29d194f1e93179d3623553461/users/bd6614797cd2436daab7be69c3d922cf/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.329193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9481f0f29d194f1e93179d3623553461, user_id=bd6614797cd2436daab7be69c3d922cf, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.332094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.348092 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1248/2493] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:59 2022] PUT /identity/v3/projects/9481f0f29d194f1e93179d3623553461/users/bd6614797cd2436daab7be69c3d922cf/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:59.352363 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.358868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:01:59.362747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0c67c-6db0-488c-a08f-e3ec863a2db8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.423305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82961d51-4c66-4612-9ef6-d323999c878e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.423842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961d51-4c66-4612-9ef6-d323999c878e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.424113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961d51-4c66-4612-9ef6-d323999c878e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.424373 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82961d51-4c66-4612-9ef6-d323999c878e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.425013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961d51-4c66-4612-9ef6-d323999c878e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.426600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82961d51-4c66-4612-9ef6-d323999c878e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.436897 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1246/2494] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:01:59.441507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.452181 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64a838-7f99-4b48-acbc-8df749fe5a70 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.457202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.457838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.458110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.458376 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] PATH_INFO: `/v3/projects/9481f0f29d194f1e93179d3623553461/users/bd6614797cd2436daab7be69c3d922cf/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.460401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9481f0f29d194f1e93179d3623553461, user_id=bd6614797cd2436daab7be69c3d922cf, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.463033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.476173 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1249/2495] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:01:59 2022] PUT /identity/v3/projects/9481f0f29d194f1e93179d3623553461/users/bd6614797cd2436daab7be69c3d922cf/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:01:59.488082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.488368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.488634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.518430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] MFA Rules not processed for user `bd6614797cd2436daab7be69c3d922cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:01:59.519923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:59.602810 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1247/2496] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:01:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:01:59.614942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.617029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:01:59.676718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5c31-95c7-404e-bd87-77c5b5a14a39 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.682236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.683091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.683586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.683928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.698034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.699911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.709976 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1250/2497] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:01:59.789807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:01:59.801906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3024a-1ec5-4e50-a9a5-95bced7ba6a9 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:01:59.806392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6c723-7953-4bfc-983d-29318ca9f0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:01:59.807001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c723-7953-4bfc-983d-29318ca9f0dd service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:01:59.807299 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c723-7953-4bfc-983d-29318ca9f0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:01:59.807561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c723-7953-4bfc-983d-29318ca9f0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:01:59.866282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6c723-7953-4bfc-983d-29318ca9f0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:01:59.867750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6c723-7953-4bfc-983d-29318ca9f0dd service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:01:59.877031 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1248/2498] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:01:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:01.458498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:01.458498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:01.518174 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d08174-0b30-4b05-992a-1a500e3956de service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:01.522298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:01.523050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:01.523447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:01.523811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:01.525631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:01.549812 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:01.598604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:01.600224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:01.610308 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1251/2499] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:01.746137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:01.746659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:01.746768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:01.769598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:01.770806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:01.790832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:01.830409 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1249/2500] 10.222.0.23 () {58 vars in 967 bytes} [Sun Sep 11 01:02:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:01.892743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:01.894177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:01.967864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676db1d-3c99-45c0-a660-9725b5023b23 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:01.973601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c037c-ea1a-47a0-a413-c42b92983df6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:01.974223 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c037c-ea1a-47a0-a413-c42b92983df6 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:01.974494 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c037c-ea1a-47a0-a413-c42b92983df6 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:01.974978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c037c-ea1a-47a0-a413-c42b92983df6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:01.987975 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c037c-ea1a-47a0-a413-c42b92983df6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:01.989730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c037c-ea1a-47a0-a413-c42b92983df6 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:02.000330 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1252/2501] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 01:02:02.792230 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:02.805199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac2391-69db-4f79-a429-eb9a6583c697 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:02.810219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:02.811082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:02.811439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:02.811711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] PATH_INFO: `/v3/users/1407fd80e2dd4516825c0d2e16016d88` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:02.812868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1407fd80e2dd4516825c0d2e16016d88)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:02.814866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:02.876246 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1407fd80e2dd4516825c0d2e16016d88'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:02.887489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1407fd80e2dd4516825c0d2e16016d88'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:02.902150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1407fd80e2dd4516825c0d2e16016d88'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:02.902954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1407fd80e2dd4516825c0d2e16016d88'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:02.910906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1407fd80e2dd4516825c0d2e16016d88'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:02.918179 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1250/2502] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:02 2022] DELETE /identity/v3/users/1407fd80e2dd4516825c0d2e16016d88 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:02.922898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:02.923453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:02.923453 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:02.943635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:02.944843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:03.011390 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1253/2503] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:03.021626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:03.023208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:03.083416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80348c1-8d87-47af-80e3-679d73158c18 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:03.087256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:03.087893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:03.088162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:03.088428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:03.098498 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:03.099965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:03.110457 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1251/2504] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:03.535741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:03.537244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:03.550365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077a7fb-5845-41a5-ad0d-172247b22cba None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:03.556110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:03.556744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:03.557011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:03.557273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] PATH_INFO: `/v3/projects/b4224af1c6304bf2ba9307b711aa2302` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:03.558392 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4224af1c6304bf2ba9307b711aa2302)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:03.560387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:03.627885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:03.653901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4224af1c6304bf2ba9307b711aa2302', 'request_id': 'req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.668045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4224af1c6304bf2ba9307b711aa2302', 'request_id': 'req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.680099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4224af1c6304bf2ba9307b711aa2302', 'request_id': 'req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.713799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] The token cache is being invalidate because project b4224af1c6304bf2ba9307b711aa2302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:03.713799 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.717572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.719866 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1254/2505] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:03 2022] DELETE /identity/v3/projects/b4224af1c6304bf2ba9307b711aa2302 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:03.724132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:03.737214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f505368-3304-4945-9b99-6979aae5b5d0 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:03.833010 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:03.876196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:03.876863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:03.877140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:03.877402 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] PATH_INFO: `/v3/users/bd6614797cd2436daab7be69c3d922cf` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:03.878860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd6614797cd2436daab7be69c3d922cf)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:03.882647 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:03.948366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd6614797cd2436daab7be69c3d922cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.958804 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd6614797cd2436daab7be69c3d922cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.974522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd6614797cd2436daab7be69c3d922cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.980916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd6614797cd2436daab7be69c3d922cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.989164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-22302603-b04f-40e4-a285-db6d328af531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd6614797cd2436daab7be69c3d922cf'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:03.999068 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1252/2506] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:03 2022] DELETE /identity/v3/users/bd6614797cd2436daab7be69c3d922cf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:04.427197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:04.430317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:04.455206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578588f-f54b-45e1-84fb-bd8c8c0dec0d admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:04.532700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:04.533447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:04.533789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:04.534125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] PATH_INFO: `/v3/projects/9481f0f29d194f1e93179d3623553461` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:04.535360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9481f0f29d194f1e93179d3623553461)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:04.538110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:04.589973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9481f0f29d194f1e93179d3623553461', 'request_id': 'req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:04.598110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9481f0f29d194f1e93179d3623553461', 'request_id': 'req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:04.613521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9481f0f29d194f1e93179d3623553461', 'request_id': 'req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:04.635852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] The token cache is being invalidate because project 9481f0f29d194f1e93179d3623553461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:04.636283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:04.636945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e57f670b-39ea-4aac-9ede-1d5ee0928ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:04.638851 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1255/2507] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:04 2022] DELETE /identity/v3/projects/9481f0f29d194f1e93179d3623553461 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:04.643678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:04.644116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:04.644614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:04.675235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:04.676757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:04.754003 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1253/2508] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:04.762626 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:04.762911 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:04.763304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:04.790280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:04.791822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:04.804085 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1256/2509] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:04.805072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:04.819236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e197f-2754-46e1-91e4-ee7f0acb6270 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:04.825825 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:04.826721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:04.827267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:04.827689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:04.828523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:04.828823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:04.830628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:04.845298 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1254/2510] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:04.849755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:04.859036 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820023b-60ac-4c39-8991-b1ba84a42183 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:04.864386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:04.865023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:04.865293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:04.865560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:04.866204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:04.869052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:04.916856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:04.917170 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1257/2511] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:04 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:04.931552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c61913b-4e99-48e8-a321-ae6153a83034 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:04.993786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:05.008080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.008788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.009434 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.009945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.010940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.014248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.055844 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1255/2512] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:04 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:05.060213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.075773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616809c3-9e6b-4793-b2d1-00ecbe5a917c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.081119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.081750 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.082155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.082427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.083225 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.085211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.101547 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1258/2513] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:05.102710 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.114969 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac4f79-8441-40ef-8806-7c5e88b8dab6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.120417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426e3aa-49f6-4d5d-beb5-598e4c1fc869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.121036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e3aa-49f6-4d5d-beb5-598e4c1fc869 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.121523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e3aa-49f6-4d5d-beb5-598e4c1fc869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.121938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e3aa-49f6-4d5d-beb5-598e4c1fc869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.122595 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426e3aa-49f6-4d5d-beb5-598e4c1fc869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.124858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6426e3aa-49f6-4d5d-beb5-598e4c1fc869 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.140216 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1256/2514] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:05.145859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.163156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd0b9-6dac-4bf0-a069-4e37507e85ec admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.171366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.172231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.172513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.173522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] PATH_INFO: `/v3/projects/3d104c50b11c463dad980eb3030fec53/users/c2b639d1e407483ba8375cf32e75996a/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.198000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d104c50b11c463dad980eb3030fec53, user_id=c2b639d1e407483ba8375cf32e75996a, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.202112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.206986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:05.237131 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1259/2515] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:05 2022] PUT /identity/v3/projects/3d104c50b11c463dad980eb3030fec53/users/c2b639d1e407483ba8375cf32e75996a/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:05.248200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.248719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.249194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.306160 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] MFA Rules not processed for user `c2b639d1e407483ba8375cf32e75996a`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:05.308796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:05.430549 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1257/2516] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:05.431479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.517191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4b941-b218-442a-959a-7ed855a076b1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.527324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.528321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.528695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.529060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.529951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.534286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.596570 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1260/2517] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:05 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:05.601022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.618589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576b8e7-0e74-46b9-bc7e-a0000c10307a None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.748661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.749510 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.749876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.750270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.750946 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.754548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.787160 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1258/2518] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:05 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:05.798076 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.807875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e5cea-1219-4bb6-a5ec-ecc8ff9b0710 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.827656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.828531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.828859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.829273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.830119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.832120 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.844671 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1261/2519] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:05.849166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.862136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfc0ad-8711-4736-8921-360f366464c2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.870807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:05.871961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:05.872346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:05.873478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] PATH_INFO: `/v3/projects/7c6c7a60c03a4107b9bc87981c70a260/users/c6bb1f1fe565403f81a2b552d815d350/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:05.908634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6c7a60c03a4107b9bc87981c70a260, user_id=c6bb1f1fe565403f81a2b552d815d350, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:05.911864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:05.932303 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1259/2520] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:05 2022] PUT /identity/v3/projects/7c6c7a60c03a4107b9bc87981c70a260/users/c6bb1f1fe565403f81a2b552d815d350/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:05.936274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:05.947315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30890ba9-a905-4e1a-86dc-c351dbb42f08 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:05.997828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.000118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.018199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.018803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.019031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.019373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.020042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.022061 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.035790 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1262/2521] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:06.036824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.037291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:06.048148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa047c8c-20f2-4b70-846f-a5fbc25a766b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.052916 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd50b7f-5998-48d0-b961-80ceeae789b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.053596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd50b7f-5998-48d0-b961-80ceeae789b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.053876 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd50b7f-5998-48d0-b961-80ceeae789b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.054149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd50b7f-5998-48d0-b961-80ceeae789b7 admin admin] PATH_INFO: `/v3/projects/7c6c7a60c03a4107b9bc87981c70a260/users/c6bb1f1fe565403f81a2b552d815d350/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.056363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd50b7f-5998-48d0-b961-80ceeae789b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6c7a60c03a4107b9bc87981c70a260, user_id=c6bb1f1fe565403f81a2b552d815d350, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.059083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd50b7f-5998-48d0-b961-80ceeae789b7 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.070041 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1263/2522] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:06 2022] PUT /identity/v3/projects/7c6c7a60c03a4107b9bc87981c70a260/users/c6bb1f1fe565403f81a2b552d815d350/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:06.076208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6eb999-8246-4ede-b38b-c39bf7cfe51b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.081693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.081980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.082251 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.110048 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] MFA Rules not processed for user `c6bb1f1fe565403f81a2b552d815d350`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:06.111322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.137235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.137864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.138172 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.138441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] PATH_INFO: `/v3/users/364cf4a7a3bf4e70973c78a5d690e998` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.139651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] RBAC: Authorizing `identity:delete_user(user_id=364cf4a7a3bf4e70973c78a5d690e998)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.142411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.152514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:06.185270 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1264/2523] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:06.195986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '364cf4a7a3bf4e70973c78a5d690e998'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.196441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.197811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.205794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '364cf4a7a3bf4e70973c78a5d690e998'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.219489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '364cf4a7a3bf4e70973c78a5d690e998'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.220184 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '364cf4a7a3bf4e70973c78a5d690e998'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.227523 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '364cf4a7a3bf4e70973c78a5d690e998'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.233587 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1260/2524] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:05 2022] DELETE /identity/v3/users/364cf4a7a3bf4e70973c78a5d690e998 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:06.259739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573856a8-ce8a-467b-bdb3-82f319514f9d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.311474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.312090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.312362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.312634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.314401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.380290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.381752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.392920 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1265/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:06.413895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.415471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.475784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:06.490301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3ebe4-f8c8-4ca0-a500-45bb9bb4b90d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.500304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.501263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.501865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.502458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] PATH_INFO: `/v3/projects/89e6e6a587374d53b8515f264dfd8cb6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.503482 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.503924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89e6e6a587374d53b8515f264dfd8cb6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.505536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.506291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.516662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78b1e9-8494-4648-a474-627030e0006f service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.520516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.521128 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.521402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.521671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.523510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.564194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89e6e6a587374d53b8515f264dfd8cb6', 'request_id': 'req-95ff41a9-1580-4d14-8d2c-d6a75f498b35'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.572570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:06.585705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89e6e6a587374d53b8515f264dfd8cb6', 'request_id': 'req-95ff41a9-1580-4d14-8d2c-d6a75f498b35'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.596188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89e6e6a587374d53b8515f264dfd8cb6', 'request_id': 'req-95ff41a9-1580-4d14-8d2c-d6a75f498b35'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.598667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.600367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60245497-32f4-46f7-9a20-c5090cccc1dd service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.609146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] The token cache is being invalidate because project 89e6e6a587374d53b8515f264dfd8cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:06.609605 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.610485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-95ff41a9-1580-4d14-8d2c-d6a75f498b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:06.612103 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1261/2526] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:06 2022] DELETE /identity/v3/projects/89e6e6a587374d53b8515f264dfd8cb6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:06.615032 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1266/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:06.616984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.617337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.617665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.641821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:06.643031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.734715 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1262/2528] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:06.742631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.743066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.743545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.764718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:06.765947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:06.776491 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1267/2529] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:06.781169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.792865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ec688c-c0c9-4958-8844-eea92cb9c302 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.798029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.798750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.799084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.799437 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.800255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:06.800729 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.802441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.819874 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1263/2530] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:06.820993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.830854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387206b-37ce-4c3b-9c31-8b15346a4a6d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.835625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.836295 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.836635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.836964 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.837675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.839705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:06.873564 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1268/2531] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:06 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:06.874711 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:06.885123 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514936d-be9e-42e1-b063-c9fb1515334b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:06.981588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:06.983231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:06.983747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:06.984032 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:06.984694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:06.986713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.005570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:07.017546 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1264/2532] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:06 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:07.018774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.029160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55965283-1ad8-42e6-a4da-be3e7945cb08 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.033920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.034525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.034801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.035067 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.035756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.037354 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.047950 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1269/2533] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:07.052305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.063094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f2c705-fe10-4ab4-88ef-82bc2c667a14 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.068271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e2718-eac6-4e63-bb3c-5af8eeb6dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.068873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e2718-eac6-4e63-bb3c-5af8eeb6dcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.069153 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e2718-eac6-4e63-bb3c-5af8eeb6dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.069426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e2718-eac6-4e63-bb3c-5af8eeb6dcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.070127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e2718-eac6-4e63-bb3c-5af8eeb6dcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.071720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e2718-eac6-4e63-bb3c-5af8eeb6dcf2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.083017 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1265/2534] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:07.087447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.097830 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b18eac-7791-42a3-8259-e54243081387 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.102776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.103438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.103716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.103991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] PATH_INFO: `/v3/projects/50351bd2356641dfbdca60393123fe96/users/274ec017ceb74c74a7c00dee2bf02a23/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.119142 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] RBAC: Authorizing `identity:create_grant(project_id=50351bd2356641dfbdca60393123fe96, user_id=274ec017ceb74c74a7c00dee2bf02a23, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.121894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.135415 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1270/2535] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:07 2022] PUT /identity/v3/projects/50351bd2356641dfbdca60393123fe96/users/274ec017ceb74c74a7c00dee2bf02a23/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:07.145849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.146146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.146378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.177660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] MFA Rules not processed for user `274ec017ceb74c74a7c00dee2bf02a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:07.178890 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:07.257143 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1266/2536] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:07.262360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.327304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71576adc-6b2d-4ef0-9c6f-56732587aaef admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.333018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.333610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.333867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.334172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.334809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.336742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.365070 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1271/2537] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:07 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:07.369479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.381601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0c036-79a4-436a-9d89-fcc538fdfb47 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.441868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.442520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.442822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.443112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.443887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.446033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.468056 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1267/2538] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:07 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:07.472337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.483297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6702e-0bd2-433d-a428-6b2f665a77e1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.489257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.489858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.490131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.490397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.491031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.493435 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.506873 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1272/2539] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:07.512337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.522961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da40b8bc-d58c-4534-8c96-e3f042a939ff admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.528295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.529008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.529308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.529572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] PATH_INFO: `/v3/projects/2731d69782434a82832d8d280e951a09/users/b9980bed44cd4fba8fe63053bfb447d4/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.545247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2731d69782434a82832d8d280e951a09, user_id=b9980bed44cd4fba8fe63053bfb447d4, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.563417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.563417 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1268/2540] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:07 2022] PUT /identity/v3/projects/2731d69782434a82832d8d280e951a09/users/b9980bed44cd4fba8fe63053bfb447d4/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:07.565831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.577824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89f675-aef0-4065-a1c0-49fc3bd70e75 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.626595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:07.630390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.632517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:07.638047 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.638856 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.639119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.639420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.640065 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.641676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.655028 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1273/2541] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:07.656087 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.666686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc064b-0ee0-410c-ba71-5051c0939b59 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.671363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26296305-4e12-4e24-8580-48f01675a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.671995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296305-4e12-4e24-8580-48f01675a855 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.672268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296305-4e12-4e24-8580-48f01675a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.672536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26296305-4e12-4e24-8580-48f01675a855 admin admin] PATH_INFO: `/v3/projects/2731d69782434a82832d8d280e951a09/users/b9980bed44cd4fba8fe63053bfb447d4/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.674464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26296305-4e12-4e24-8580-48f01675a855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2731d69782434a82832d8d280e951a09, user_id=b9980bed44cd4fba8fe63053bfb447d4, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.677231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26296305-4e12-4e24-8580-48f01675a855 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.680993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:07.688249 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1274/2542] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:07 2022] PUT /identity/v3/projects/2731d69782434a82832d8d280e951a09/users/b9980bed44cd4fba8fe63053bfb447d4/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:07.693788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a35702-d5d8-4046-9d88-d0a2ef35b0a4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.700995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.701302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.701584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.732724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] MFA Rules not processed for user `b9980bed44cd4fba8fe63053bfb447d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:07.733934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:07.749203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.750254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.750641 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.750802 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.753207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:07.816543 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1275/2543] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:07.823026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.824616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.829422 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:07.830766 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:07.835118 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1269/2544] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:07.844032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9140a5a-2205-4eac-8a37-8118473feb21 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:07.847778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e8a98b-a37e-4e40-8b37-58b307160ee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:07.848497 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8a98b-a37e-4e40-8b37-58b307160ee7 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:07.848838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8a98b-a37e-4e40-8b37-58b307160ee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:07.849190 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e8a98b-a37e-4e40-8b37-58b307160ee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:07.860458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8a98b-a37e-4e40-8b37-58b307160ee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:07.862001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e8a98b-a37e-4e40-8b37-58b307160ee7 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:07.872559 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1276/2545] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:08.460756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:08.533671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92736d-23f3-42f7-b9ab-55adb02b972e service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:08.539497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:08.540125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:08.540477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:08.540747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] PATH_INFO: `/v3/users/274ec017ceb74c74a7c00dee2bf02a23` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:08.541914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=274ec017ceb74c74a7c00dee2bf02a23)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:08.543950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:08.604758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '274ec017ceb74c74a7c00dee2bf02a23'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:08.613234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:08.621619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '274ec017ceb74c74a7c00dee2bf02a23'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:08.639576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '274ec017ceb74c74a7c00dee2bf02a23'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:08.640565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '274ec017ceb74c74a7c00dee2bf02a23'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:08.647968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '274ec017ceb74c74a7c00dee2bf02a23'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:08.654347 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1270/2546] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:08 2022] DELETE /identity/v3/users/274ec017ceb74c74a7c00dee2bf02a23 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:08.659002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:08.659542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:08.659640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:08.680324 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:08.681528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:08.743465 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1277/2547] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:08.753428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:08.754877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:08.813309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5c9e4-e8f6-4ce9-a6cd-6bcf7021ba94 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:08.816907 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:08.817513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:08.817791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:08.818054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:08.827308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:08.828737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:08.838897 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1271/2548] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:09.120378 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.122011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.133984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd38303-e98b-4ba7-b81b-60f6a258ea28 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.138590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.139260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.139541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.139813 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] PATH_INFO: `/v3/projects/50351bd2356641dfbdca60393123fe96` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.144021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50351bd2356641dfbdca60393123fe96)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:09.146028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:09.188582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:09.195745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50351bd2356641dfbdca60393123fe96', 'request_id': 'req-511c19ed-66c0-4354-96b4-07a4cf61b777'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.205970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50351bd2356641dfbdca60393123fe96', 'request_id': 'req-511c19ed-66c0-4354-96b4-07a4cf61b777'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.212525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50351bd2356641dfbdca60393123fe96', 'request_id': 'req-511c19ed-66c0-4354-96b4-07a4cf61b777'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.223768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] The token cache is being invalidate because project 50351bd2356641dfbdca60393123fe96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:09.224052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.224761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.226353 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1278/2549] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:09 2022] DELETE /identity/v3/projects/50351bd2356641dfbdca60393123fe96 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:09.230560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.240467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285076d2-4f07-4dd8-a2eb-66249c91fbdd service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.303717 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.304355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.304626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.305005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] PATH_INFO: `/v3/users/b9980bed44cd4fba8fe63053bfb447d4` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.306239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9980bed44cd4fba8fe63053bfb447d4)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:09.308207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:09.359424 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.360661 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9980bed44cd4fba8fe63053bfb447d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.361207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.368280 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9980bed44cd4fba8fe63053bfb447d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.375655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c19ed-66c0-4354-96b4-07a4cf61b777 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.380564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.381036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9980bed44cd4fba8fe63053bfb447d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.381191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.381462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.381718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9980bed44cd4fba8fe63053bfb447d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.381828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] PATH_INFO: `/v3/users/c2b639d1e407483ba8375cf32e75996a` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.382979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:09.383784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2b639d1e407483ba8375cf32e75996a)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:09.385875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:09.391106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-a994f1ab-0054-40e7-9d1b-5f4807ee0a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9980bed44cd4fba8fe63053bfb447d4'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.398616 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1272/2550] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:09 2022] DELETE /identity/v3/users/b9980bed44cd4fba8fe63053bfb447d4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:09.435152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2b639d1e407483ba8375cf32e75996a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.443863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2b639d1e407483ba8375cf32e75996a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.451660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2b639d1e407483ba8375cf32e75996a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.452434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2b639d1e407483ba8375cf32e75996a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.465659 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2b639d1e407483ba8375cf32e75996a'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.472008 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1279/2551] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:09 2022] DELETE /identity/v3/users/c2b639d1e407483ba8375cf32e75996a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:09.476864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.477400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.477400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.497631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:09.498842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.564541 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1273/2552] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:09.574487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.576054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.644538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e09d3a-a2ec-4578-8ffc-0191ef9eaef0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.648244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37f801-f1dc-4679-880d-a9591123bbb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.648842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37f801-f1dc-4679-880d-a9591123bbb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.649114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37f801-f1dc-4679-880d-a9591123bbb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.649381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37f801-f1dc-4679-880d-a9591123bbb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.658983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37f801-f1dc-4679-880d-a9591123bbb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:09.660523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37f801-f1dc-4679-880d-a9591123bbb4 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:09.670872 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1280/2553] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:09.689771 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.691870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.706919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022c42e-9f87-421d-ad07-168cc6ac47cf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.712494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.713293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.713681 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.714044 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] PATH_INFO: `/v3/projects/2731d69782434a82832d8d280e951a09` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.715512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2731d69782434a82832d8d280e951a09)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:09.718251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:09.769782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2731d69782434a82832d8d280e951a09', 'request_id': 'req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.778085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2731d69782434a82832d8d280e951a09', 'request_id': 'req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.784289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2731d69782434a82832d8d280e951a09', 'request_id': 'req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.796135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] The token cache is being invalidate because project 2731d69782434a82832d8d280e951a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:09.796419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.797083 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2aa5ed18-58a3-49e1-8c2f-3fdd4ad188dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:09.798847 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1274/2554] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:09 2022] DELETE /identity/v3/projects/2731d69782434a82832d8d280e951a09 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:09.803546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.803829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.804108 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.825012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:09.826213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.878948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:09.899663 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1281/2555] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:09.907521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.907820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.908092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.929410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:09.930696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:09.940473 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1275/2556] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:09.945091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.954850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582cc14e-2099-4669-8295-9772ce10e7f3 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.959778 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.960382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.960651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.960894 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.961552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:09.961847 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:09.963463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:09.980889 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:09.987025 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1282/2557] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:09.991810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2e7a-72b4-4094-9594-6a71b944a788 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:09.996534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:09.997141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:09.997410 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:09.997677 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:09.998291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.000255 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.040513 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1276/2558] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:09 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:10.047426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.047573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.052535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:10.067572 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b7ba0-0a48-4621-8d1e-49e16a15595e admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.098317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:10.117225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8fadd-4be6-4054-bd80-87db9ba10008 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.123168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.123416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.123663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.123934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.124687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.126911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.127993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.128814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.129263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.129702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] PATH_INFO: `/v3/projects/3d104c50b11c463dad980eb3030fec53` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.131474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d104c50b11c463dad980eb3030fec53)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.134042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.167702 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1277/2559] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:10 2022] POST /identity/v3/users => generated 290 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:10.169235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.179887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9590542-d701-4b44-963a-38a8dc2ce7b1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.193670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d104c50b11c463dad980eb3030fec53', 'request_id': 'req-9711c46e-75ba-46db-bffe-64e8611b670f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.203513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d104c50b11c463dad980eb3030fec53', 'request_id': 'req-9711c46e-75ba-46db-bffe-64e8611b670f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.211948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d104c50b11c463dad980eb3030fec53', 'request_id': 'req-9711c46e-75ba-46db-bffe-64e8611b670f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.225635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] The token cache is being invalidate because project 3d104c50b11c463dad980eb3030fec53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:10.225923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.226857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.228614 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1283/2560] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:10 2022] DELETE /identity/v3/projects/3d104c50b11c463dad980eb3030fec53 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:10.232377 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.234157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:10.245199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.245834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.246111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.246404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.247060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.248700 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.259411 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1278/2561] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:10.263536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.265881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:10.279314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596baa96-1cd7-41e9-9d52-4808f37a2df8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.284284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.284901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.285175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.285444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.286092 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.288141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.299759 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1279/2562] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:10.304137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.313504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711c46e-75ba-46db-bffe-64e8611b670f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.314365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dfd7f9-2217-4bdf-8ded-9b355c721e5a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.323697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.324442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.324814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.325151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] PATH_INFO: `/v3/users/c6bb1f1fe565403f81a2b552d815d350` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.326374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6bb1f1fe565403f81a2b552d815d350)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.328673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.332426 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:10.332786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e474ae-75b7-4ef2-911d-3089e6351c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.333540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e474ae-75b7-4ef2-911d-3089e6351c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.334634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e474ae-75b7-4ef2-911d-3089e6351c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.335086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e474ae-75b7-4ef2-911d-3089e6351c06 admin admin] PATH_INFO: `/v3/projects/c5e46682fc414d03a31f78f41b85211a/users/eddd278790044c7cac4c41d3b4d9cac1/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.357515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e474ae-75b7-4ef2-911d-3089e6351c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e46682fc414d03a31f78f41b85211a, user_id=eddd278790044c7cac4c41d3b4d9cac1, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.360192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e474ae-75b7-4ef2-911d-3089e6351c06 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.376451 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1280/2563] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:10 2022] PUT /identity/v3/projects/c5e46682fc414d03a31f78f41b85211a/users/eddd278790044c7cac4c41d3b4d9cac1/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:10.386945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.387258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.387528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.393289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6bb1f1fe565403f81a2b552d815d350'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.404767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6bb1f1fe565403f81a2b552d815d350'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.413282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6bb1f1fe565403f81a2b552d815d350'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.413884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6bb1f1fe565403f81a2b552d815d350'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.418734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] MFA Rules not processed for user `eddd278790044c7cac4c41d3b4d9cac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:10.420036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:10.427289 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6bb1f1fe565403f81a2b552d815d350'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.434046 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1284/2564] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:10 2022] DELETE /identity/v3/users/c6bb1f1fe565403f81a2b552d815d350 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:10.498343 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1281/2565] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:10.503430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.505229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:10.581423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6403ca-7093-41e1-a7da-1af9c877f9e3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.587380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.588082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.588451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.588823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.589562 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.591825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.629354 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1285/2566] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:10 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:10.631099 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.666616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:10.711435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f0b893-cf79-47ea-9c94-c030aa82c3c7 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.739344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.741322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:10.796105 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.796725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.796999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.797266 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.797900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.800000 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.831664 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1282/2567] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:10 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:10.838084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.849783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc0a83-99ba-4f81-a2b6-8d931a64bbda admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.859680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.860799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.861518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.861947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.863037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.864897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.874542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93205e65-509e-4948-b0de-91c33bf280d9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.878518 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1283/2568] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:10.879393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.880022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.880299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.880565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] PATH_INFO: `/v3/projects/7c6c7a60c03a4107b9bc87981c70a260` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.881698 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6c7a60c03a4107b9bc87981c70a260)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.882687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.883630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.892136 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:10.894192 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dea9f-bf3c-458d-aa12-bc2a7f8a5185 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:10.899655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:10.900314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.900585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.900852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] PATH_INFO: `/v3/projects/5cecb0d73dd94896980aa39163fc2288/users/47c9689567774872a86b85d59c6d055c/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.917960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cecb0d73dd94896980aa39163fc2288, user_id=47c9689567774872a86b85d59c6d055c, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:10.920928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:10.934567 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1284/2569] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:10 2022] PUT /identity/v3/projects/5cecb0d73dd94896980aa39163fc2288/users/47c9689567774872a86b85d59c6d055c/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:10.938589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:10.940768 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6c7a60c03a4107b9bc87981c70a260', 'request_id': 'req-cf19f057-8c54-4869-af59-28393ab0dfc1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.951069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6c7a60c03a4107b9bc87981c70a260', 'request_id': 'req-cf19f057-8c54-4869-af59-28393ab0dfc1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.958534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6c7a60c03a4107b9bc87981c70a260', 'request_id': 'req-cf19f057-8c54-4869-af59-28393ab0dfc1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.970461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] The token cache is being invalidate because project 7c6c7a60c03a4107b9bc87981c70a260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:10.970740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.971423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cf19f057-8c54-4869-af59-28393ab0dfc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:10.972982 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1286/2570] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:10 2022] DELETE /identity/v3/projects/7c6c7a60c03a4107b9bc87981c70a260 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:10.977702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4da3b-60af-486a-ba49-dfd23aeca496 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:10.977984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4da3b-60af-486a-ba49-dfd23aeca496 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:10.978246 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4da3b-60af-486a-ba49-dfd23aeca496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:10.998022 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ebc4da3b-60af-486a-ba49-dfd23aeca496 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:10.999900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ebc4da3b-60af-486a-ba49-dfd23aeca496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.004574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddade908-1739-49a3-ab63-eacbdfc87a48 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.013858 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1287/2571] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:11.014064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.014064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.014064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.014064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.014433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.014433 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.016100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.016444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.016816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.026076 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1285/2572] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:11.027399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.029372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.037239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:11.038882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.044413 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521a49-38f3-4397-a46a-270bf9c2203f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.048699 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1288/2573] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:11.049930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e67e39-cfef-40e5-9a2a-dae8dc2b79dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.050760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67e39-cfef-40e5-9a2a-dae8dc2b79dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.051037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67e39-cfef-40e5-9a2a-dae8dc2b79dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.052969 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e67e39-cfef-40e5-9a2a-dae8dc2b79dd admin admin] PATH_INFO: `/v3/projects/5cecb0d73dd94896980aa39163fc2288/users/47c9689567774872a86b85d59c6d055c/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.054142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e67e39-cfef-40e5-9a2a-dae8dc2b79dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cecb0d73dd94896980aa39163fc2288, user_id=47c9689567774872a86b85d59c6d055c, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.054368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.057069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e67e39-cfef-40e5-9a2a-dae8dc2b79dd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.066810 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e927cace-949b-459e-8a9d-4d5a575f0882 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.071358 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1286/2574] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:11 2022] PUT /identity/v3/projects/5cecb0d73dd94896980aa39163fc2288/users/47c9689567774872a86b85d59c6d055c/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:11.084079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.085049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.086235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.129731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:11.136519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.137176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.137452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.137722 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.138375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:11.138867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.144718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.150013 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] MFA Rules not processed for user `47c9689567774872a86b85d59c6d055c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:11.152424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.156354 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1289/2575] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:11.164126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.187899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42069db9-5eb3-4c7f-9ea5-0dbbb3023d5f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.193267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.193879 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.194543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.194815 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.195858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.197954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.233145 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1290/2576] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/projects => generated 381 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:11.237449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.244810 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1287/2577] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:11.256966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.258720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.296248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:11.301177 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a795-e912-4aa8-9ef7-9c9e024ab90b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.307430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.308157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.308612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.308970 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.309835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.312121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.319100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7affe0c-7b0b-4acd-90b9-09858f29ae16 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.340481 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1291/2578] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:11.341947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.353930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84019368-19e3-405f-899a-b7121fc46ca0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.359522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.360197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.360471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.361069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.361488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.363381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.372310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.372951 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.373231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.373501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.375164 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1292/2579] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:11.378569 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.379294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.390074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662973d-7e53-49e2-86ff-44c250e13c9f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.397757 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.398431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.398779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.399093 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.400888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.403416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.416353 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1293/2580] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:11.420556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.431035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8c660-a813-4e07-91ff-558f22c00157 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.436235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.436962 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.437305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.437638 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] PATH_INFO: `/v3/projects/b7cc6f393be84a9a9c0a366efb8ce16e/users/6b07ca77fd4e4ae888d4e71f2fbc482c/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.448782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.450251 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a242baa-868c-4239-8b19-a797bfa87773 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.452947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7cc6f393be84a9a9c0a366efb8ce16e, user_id=6b07ca77fd4e4ae888d4e71f2fbc482c, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.455619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.462120 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1288/2581] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:11.476934 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1294/2582] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:11 2022] PUT /identity/v3/projects/b7cc6f393be84a9a9c0a366efb8ce16e/users/6b07ca77fd4e4ae888d4e71f2fbc482c/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:11.489371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.489608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.489781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.498009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:11.530694 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] MFA Rules not processed for user `6b07ca77fd4e4ae888d4e71f2fbc482c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:11.532761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:11.613935 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1289/2583] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:11.617518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.691588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566555e-7229-43cd-b0be-494cbbc91aad admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.697774 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.698576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.699345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.699726 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.700592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.703100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.739378 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1295/2584] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:11.741051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.758990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dedf4d3-ce74-4088-a783-bc09858fefaa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.829926 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.830544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.830807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.831076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.831785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.833817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.858310 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1290/2585] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:11 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:11.862365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.872967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cba92-a2d2-4f6c-8efa-d2d518464f7c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.877580 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.878179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.878456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.878728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.879408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.881121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.891894 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1296/2586] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:11.896125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.906399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c996ae-a111-43de-b52a-427cebd17fe6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:11.911578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:11.912211 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:11.912844 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:11.913113 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] PATH_INFO: `/v3/projects/459f9926231a43a892885dbaf5fd0034/users/e92dfa6c34ca47348652928cdc82cbd5/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:11.928698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459f9926231a43a892885dbaf5fd0034, user_id=e92dfa6c34ca47348652928cdc82cbd5, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:11.931428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:11.952106 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1291/2587] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:11 2022] PUT /identity/v3/projects/459f9926231a43a892885dbaf5fd0034/users/e92dfa6c34ca47348652928cdc82cbd5/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:11.958616 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:11.971519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297f818-0aed-4530-b651-58885da1a53a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:12.028512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e60558-bd4b-4427-96d8-009852a8473c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:12.029098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60558-bd4b-4427-96d8-009852a8473c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.029382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60558-bd4b-4427-96d8-009852a8473c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.029636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e60558-bd4b-4427-96d8-009852a8473c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.030273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e60558-bd4b-4427-96d8-009852a8473c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:12.031919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e60558-bd4b-4427-96d8-009852a8473c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:12.042716 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1297/2588] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:12.048457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:12.059608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7829c3-0407-4cc6-b4a3-dadd01c9a336 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:12.064521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:12.065164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.065435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.065698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] PATH_INFO: `/v3/projects/459f9926231a43a892885dbaf5fd0034/users/e92dfa6c34ca47348652928cdc82cbd5/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.067740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459f9926231a43a892885dbaf5fd0034, user_id=e92dfa6c34ca47348652928cdc82cbd5, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:12.070347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:12.080855 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1292/2589] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:12 2022] PUT /identity/v3/projects/459f9926231a43a892885dbaf5fd0034/users/e92dfa6c34ca47348652928cdc82cbd5/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:12.091226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.091543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.091816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.113229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:12.123545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] MFA Rules not processed for user `e92dfa6c34ca47348652928cdc82cbd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:12.125516 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:12.210511 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1298/2590] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:12.224027 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:12.282324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:12.289716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e043f30-a020-4b29-a119-af4fecb5f9b3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:12.295585 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:12.296197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.296616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.296897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.307150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:12.308774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:12.319020 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1293/2591] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:12.718584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:12.782648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05f91e-6316-4de4-8cae-57ed174cc8a9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:12.788802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:12.789748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.790110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.790471 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] PATH_INFO: `/v3/users/6b07ca77fd4e4ae888d4e71f2fbc482c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.792431 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b07ca77fd4e4ae888d4e71f2fbc482c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:12.795509 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:12.847010 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b07ca77fd4e4ae888d4e71f2fbc482c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:12.858842 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b07ca77fd4e4ae888d4e71f2fbc482c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:12.865296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:12.866849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:12.868583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b07ca77fd4e4ae888d4e71f2fbc482c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:12.869256 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b07ca77fd4e4ae888d4e71f2fbc482c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:12.879014 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad569f-afe3-47b9-bcd9-8bd844e88690 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:12.882794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:12.883436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.883555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a615c65b-8305-48ff-9789-53e09e42eede admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b07ca77fd4e4ae888d4e71f2fbc482c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:12.883707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.883973 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.885663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:12.889842 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1299/2592] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:12 2022] DELETE /identity/v3/users/6b07ca77fd4e4ae888d4e71f2fbc482c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:12.894234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:12.894517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:12.894789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:12.913967 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:12.915199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:12.924345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:12.953089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:12.954580 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:12.965114 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1294/2593] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:12.979148 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1300/2594] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:12.989451 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:12.990860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:13.047362 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686025c3-06e5-4d24-9a9c-232d5e3c529a service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:13.098801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:13.099447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:13.099719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:13.100107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:13.110415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:13.111941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:13.122353 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1295/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:13.395755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:13.397254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:13.411838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69af0d2-8af0-4e75-8104-34055d571ece None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:13.417598 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:13.419282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:13.419631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:13.419822 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] PATH_INFO: `/v3/projects/b7cc6f393be84a9a9c0a366efb8ce16e` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:13.421066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7cc6f393be84a9a9c0a366efb8ce16e)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:13.423002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:13.476131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7cc6f393be84a9a9c0a366efb8ce16e', 'request_id': 'req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.484601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7cc6f393be84a9a9c0a366efb8ce16e', 'request_id': 'req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.490873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7cc6f393be84a9a9c0a366efb8ce16e', 'request_id': 'req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.502651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] The token cache is being invalidate because project b7cc6f393be84a9a9c0a366efb8ce16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:13.502959 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.503693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.505284 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1301/2596] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:13 2022] DELETE /identity/v3/projects/b7cc6f393be84a9a9c0a366efb8ce16e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:13.509565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:13.520311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe56a0d-c3b7-44d3-adc8-7f236d66a220 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:13.563762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:13.580146 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:13.580783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:13.581055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:13.581321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] PATH_INFO: `/v3/users/e92dfa6c34ca47348652928cdc82cbd5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:13.582428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92dfa6c34ca47348652928cdc82cbd5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:13.584367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:13.634799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92dfa6c34ca47348652928cdc82cbd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.642689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92dfa6c34ca47348652928cdc82cbd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.655502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92dfa6c34ca47348652928cdc82cbd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.656361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92dfa6c34ca47348652928cdc82cbd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.663864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-32a4f9cf-18d5-43a5-a4bb-cf1e0eec2f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92dfa6c34ca47348652928cdc82cbd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:13.669675 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1296/2597] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:13 2022] DELETE /identity/v3/users/e92dfa6c34ca47348652928cdc82cbd5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:13.964374 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:13.966058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:13.983857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ebb1e-25b3-41f3-93cd-a1b311c2168c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.042493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.043098 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.043568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.043843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] PATH_INFO: `/v3/projects/459f9926231a43a892885dbaf5fd0034` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.045013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459f9926231a43a892885dbaf5fd0034)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.046913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.088640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:14.097449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459f9926231a43a892885dbaf5fd0034', 'request_id': 'req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:14.105358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459f9926231a43a892885dbaf5fd0034', 'request_id': 'req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:14.111716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459f9926231a43a892885dbaf5fd0034', 'request_id': 'req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:14.122458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] The token cache is being invalidate because project 459f9926231a43a892885dbaf5fd0034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:14.122745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:14.123413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f6e96cfb-07cd-4c6e-bb0f-ad22fb503954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:14.124988 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1302/2598] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:13 2022] DELETE /identity/v3/projects/459f9926231a43a892885dbaf5fd0034 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:14.130730 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.131256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.131358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.154801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:14.156046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:14.221846 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1297/2599] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:14.225887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.226163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.226442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.246515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:14.247749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:14.255937 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1303/2600] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:14.260137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.269742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76682335-67bb-4e7a-ab0f-beef24545834 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.274317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.274916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.275208 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.275483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.276125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:14.276424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.278079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.290279 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1298/2601] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:14.294859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.304363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845603d-8d85-4959-9f8e-c837af57fd0d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.309677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.310275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.310537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.310817 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.311485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.313369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.342820 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1304/2602] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:14.347326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.358956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d06fdf-62f8-4550-a4ef-cbdb7eb59bcd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.406651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:14.421407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.422149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.422535 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.422857 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.423644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.435415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.459911 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1299/2603] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:14.461317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.472478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11da702f-63f5-4f2d-a26e-e0d19ed29910 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.477173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.477884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.478226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.478613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.479633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.481129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.491575 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1305/2604] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:14.495659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.505902 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82847b8e-7940-474f-8799-d181b57f7b38 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.510975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd6d14-8d02-4124-815a-a699b83be4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.511602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6d14-8d02-4124-815a-a699b83be4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.511990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6d14-8d02-4124-815a-a699b83be4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.512260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd6d14-8d02-4124-815a-a699b83be4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.512919 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd6d14-8d02-4124-815a-a699b83be4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.514496 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd6d14-8d02-4124-815a-a699b83be4a9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.524958 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1300/2605] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:14.529156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.539552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568545b-11eb-4efc-8db5-92526c3b0b03 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.545261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.545829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.546105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.546375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] PATH_INFO: `/v3/projects/25c4f8d37471472aabae169bddc1231c/users/f61d2e988e4f404eb9608d084eabe0ff/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.561841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c4f8d37471472aabae169bddc1231c, user_id=f61d2e988e4f404eb9608d084eabe0ff, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.564533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.578550 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1306/2606] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:14 2022] PUT /identity/v3/projects/25c4f8d37471472aabae169bddc1231c/users/f61d2e988e4f404eb9608d084eabe0ff/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:14.590076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.590430 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.590698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.620760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] MFA Rules not processed for user `f61d2e988e4f404eb9608d084eabe0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:14.621965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:14.701180 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1301/2607] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:14.705863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.769222 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7764ac-1acb-4a57-9bd9-00ef5e9c4e39 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.773900 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.774496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.774767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.775030 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.775681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.777770 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.810402 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1307/2608] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:14.814671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.826163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1b4ac-3425-4e59-bdae-d6725d9cff6c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.885975 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.886597 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.886869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.887133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.887899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.889899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.912756 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1302/2609] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:14 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:14.916871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.928523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fb07d-4be0-484a-a7fc-280faa970b56 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.933444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.934227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.934368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.934631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.935449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.937034 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:14.951102 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1308/2610] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:14.954283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:14.966349 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96400ad3-1f50-4bd1-874e-6344a6a1e9d2 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:14.972877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:14.973731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:14.974236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:14.974511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] PATH_INFO: `/v3/projects/a39b66f716c0419ca1b36c272797f3a6/users/78753964646b40089043d3562af39e4d/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:14.995831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39b66f716c0419ca1b36c272797f3a6, user_id=78753964646b40089043d3562af39e4d, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:14.998734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:15.018517 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1303/2611] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:14 2022] PUT /identity/v3/projects/a39b66f716c0419ca1b36c272797f3a6/users/78753964646b40089043d3562af39e4d/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:15.020094 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:15.035414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c361788-9ade-4fbb-8ded-f874be799dec admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:15.092672 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:15.130247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bede826-42f4-42e3-8f07-5c6899644364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:15.131753 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede826-42f4-42e3-8f07-5c6899644364 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:15.132192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede826-42f4-42e3-8f07-5c6899644364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:15.132627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bede826-42f4-42e3-8f07-5c6899644364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:15.133475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bede826-42f4-42e3-8f07-5c6899644364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:15.135991 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bede826-42f4-42e3-8f07-5c6899644364 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:15.148533 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1309/2612] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:15.153512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:15.171519 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b7388-c96d-4acc-9721-93889b983d2e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:15.175670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:15.176483 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:15.176819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:15.177195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] PATH_INFO: `/v3/projects/a39b66f716c0419ca1b36c272797f3a6/users/78753964646b40089043d3562af39e4d/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:15.179906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39b66f716c0419ca1b36c272797f3a6, user_id=78753964646b40089043d3562af39e4d, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:15.183392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:15.202530 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1304/2613] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:15 2022] PUT /identity/v3/projects/a39b66f716c0419ca1b36c272797f3a6/users/78753964646b40089043d3562af39e4d/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:15.213537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:15.213901 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:15.214241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:15.247404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] MFA Rules not processed for user `78753964646b40089043d3562af39e4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:15.248977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:15.318083 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1310/2614] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:15.329984 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:15.331879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:15.343292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:15.396426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99384ad8-05b3-4df2-b1e6-d742d7a9b502 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:15.400478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a500a-9bcc-422e-9a20-278883c221ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:15.401079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a500a-9bcc-422e-9a20-278883c221ba service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:15.401347 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a500a-9bcc-422e-9a20-278883c221ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:15.401611 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a500a-9bcc-422e-9a20-278883c221ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:15.472998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a500a-9bcc-422e-9a20-278883c221ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:15.475028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a500a-9bcc-422e-9a20-278883c221ba service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:15.486960 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1305/2615] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:16.765470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:16.766886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:16.837144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87c1063-0c63-4a9d-a43e-6ab0955709d8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:16.841992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:16.842618 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:16.842889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:16.843151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] PATH_INFO: `/v3/users/eddd278790044c7cac4c41d3b4d9cac1` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:16.844317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eddd278790044c7cac4c41d3b4d9cac1)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:16.846202 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:16.899329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eddd278790044c7cac4c41d3b4d9cac1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:16.906945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eddd278790044c7cac4c41d3b4d9cac1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:16.912669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:16.916525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eddd278790044c7cac4c41d3b4d9cac1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:16.918110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eddd278790044c7cac4c41d3b4d9cac1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:16.932489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eddd278790044c7cac4c41d3b4d9cac1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:16.939268 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1311/2616] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:16 2022] DELETE /identity/v3/users/eddd278790044c7cac4c41d3b4d9cac1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:16.943698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:16.944178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:16.944262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:16.964029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:16.965350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:17.041326 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1306/2617] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:17.049040 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:17.050447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:17.109413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fe452-6ddd-426e-bea7-7f490cbd688b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:17.113803 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:17.114468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:17.114747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:17.115071 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:17.126811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:17.128396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:17.138484 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1312/2618] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:17.429270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:17.430419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:17.445568 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a363f67-b2e2-4341-9d43-0b0ee3ec3ec6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:17.450898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:17.451578 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:17.451853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:17.452864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] PATH_INFO: `/v3/projects/c5e46682fc414d03a31f78f41b85211a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:17.454372 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e46682fc414d03a31f78f41b85211a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:17.456310 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:17.505030 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e46682fc414d03a31f78f41b85211a', 'request_id': 'req-b1997135-7fe9-482d-b644-292ef298174d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.513743 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e46682fc414d03a31f78f41b85211a', 'request_id': 'req-b1997135-7fe9-482d-b644-292ef298174d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.515933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:17.520460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e46682fc414d03a31f78f41b85211a', 'request_id': 'req-b1997135-7fe9-482d-b644-292ef298174d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.533263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] The token cache is being invalidate because project c5e46682fc414d03a31f78f41b85211a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:17.533563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.534622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.537147 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1307/2619] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:17 2022] DELETE /identity/v3/projects/c5e46682fc414d03a31f78f41b85211a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:17.541392 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:17.551138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea82739-160f-4b28-a844-d8140bf6415e service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:17.614092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:17.614738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:17.615015 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:17.615330 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] PATH_INFO: `/v3/users/47c9689567774872a86b85d59c6d055c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:17.616485 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c9689567774872a86b85d59c6d055c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:17.618427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:17.666053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c9689567774872a86b85d59c6d055c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.673621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c9689567774872a86b85d59c6d055c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.681204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c9689567774872a86b85d59c6d055c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.681857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c9689567774872a86b85d59c6d055c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.695904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c9689567774872a86b85d59c6d055c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:17.698429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:17.704722 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1313/2620] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:17 2022] DELETE /identity/v3/users/47c9689567774872a86b85d59c6d055c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:17.987394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:17.988968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.002875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1997135-7fe9-482d-b644-292ef298174d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.063419 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.064181 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.064502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.064831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] PATH_INFO: `/v3/projects/5cecb0d73dd94896980aa39163fc2288` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.064944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.065969 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cecb0d73dd94896980aa39163fc2288)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.066660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.067887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.081827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f00ae-e8a8-4dcf-8d92-8d59f98ab0f9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.087447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.088079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.088370 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.088635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] PATH_INFO: `/v3/users/f61d2e988e4f404eb9608d084eabe0ff` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.090175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61d2e988e4f404eb9608d084eabe0ff)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.092583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.120986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cecb0d73dd94896980aa39163fc2288', 'request_id': 'req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.128549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cecb0d73dd94896980aa39163fc2288', 'request_id': 'req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.139352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cecb0d73dd94896980aa39163fc2288', 'request_id': 'req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.151192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61d2e988e4f404eb9608d084eabe0ff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.154298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] The token cache is being invalidate because project 5cecb0d73dd94896980aa39163fc2288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:18.154635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.155314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b8db010a-89ff-41c4-b7c8-a9f6d3816e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.157078 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1308/2621] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:17 2022] DELETE /identity/v3/projects/5cecb0d73dd94896980aa39163fc2288 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:18.162634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d69e-74c3-4e5c-a7b4-c44b2a4e9183 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.162918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d69e-74c3-4e5c-a7b4-c44b2a4e9183 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.163214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d69e-74c3-4e5c-a7b4-c44b2a4e9183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.163459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61d2e988e4f404eb9608d084eabe0ff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.171482 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61d2e988e4f404eb9608d084eabe0ff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.172195 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61d2e988e4f404eb9608d084eabe0ff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.184198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2268d69e-74c3-4e5c-a7b4-c44b2a4e9183 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:18.185432 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2268d69e-74c3-4e5c-a7b4-c44b2a4e9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.185747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-5007d42a-0228-4a09-a1eb-4f4ee01284af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61d2e988e4f404eb9608d084eabe0ff'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.192348 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1314/2622] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:18 2022] DELETE /identity/v3/users/f61d2e988e4f404eb9608d084eabe0ff => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:18.196678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.197002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.197280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.216739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:18.218012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.252611 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1309/2623] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:18.261357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.261688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.261956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.282905 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1315/2624] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:18.285055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:18.286420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.293100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.294569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.295250 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1310/2625] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:18.298891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.308797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b406a-6c81-4eae-8300-39ade02906b1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.313516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.314110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.314386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.314655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.315371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:18.315674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.317272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.324168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:18.330203 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1311/2626] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:18.335271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.345008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceefdf-908a-4bf1-81a0-555c8b494b0d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.349654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.350237 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.350509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.350772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.351423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.353290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.356268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f47e8f0-a904-43e8-98a7-bc3a5a73b47b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.360401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.361224 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.361342 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.361538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.373097 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.374861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.383641 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1312/2627] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:18.384899 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1316/2628] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:18.387703 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.455781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be732b-5951-49e7-b5b7-2016086d0edd admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.460921 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.461685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.462020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.462316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.463129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.465139 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.488945 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1313/2629] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:18.495090 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.508029 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57d582-6323-4234-abf1-dc7569b45df8 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.512792 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.513401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.513678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.513944 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.514582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.516214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.526537 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1317/2630] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:18.530965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.544985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cd540-3ad0-4eca-b6d4-449bd16de35d admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.549996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a328a4-f652-4030-8770-7fd4ab9c7058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.550685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a328a4-f652-4030-8770-7fd4ab9c7058 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.550956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a328a4-f652-4030-8770-7fd4ab9c7058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.551250 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a328a4-f652-4030-8770-7fd4ab9c7058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.551888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a328a4-f652-4030-8770-7fd4ab9c7058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.553465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a328a4-f652-4030-8770-7fd4ab9c7058 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.564071 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1314/2631] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:18.568209 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.575853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:18.580408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729ae51-5e7a-4050-a848-18a6b7945cc0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.588326 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.589237 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.589510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.589782 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] PATH_INFO: `/v3/projects/8732fd52697240218cc40b11accb34e3/users/14241cfb17ab42b1a027c1f436ad1ada/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.605211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8732fd52697240218cc40b11accb34e3, user_id=14241cfb17ab42b1a027c1f436ad1ada, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.607986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.624147 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1318/2632] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:18 2022] PUT /identity/v3/projects/8732fd52697240218cc40b11accb34e3/users/14241cfb17ab42b1a027c1f436ad1ada/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:18.635876 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.636252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.636583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.666997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] MFA Rules not processed for user `14241cfb17ab42b1a027c1f436ad1ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:18.668304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.673104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.674850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.736145 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1315/2633] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:18.740745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.753526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7d376-75da-4412-8ca5-5eb4ba972cca admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.758080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.758711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.758987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.759282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] PATH_INFO: `/v3/projects/25c4f8d37471472aabae169bddc1231c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.760444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c4f8d37471472aabae169bddc1231c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.762393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.786426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:18.805469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfcd5ed-e788-4501-92b5-43319a944f8d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.814020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c4f8d37471472aabae169bddc1231c', 'request_id': 'req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.822255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c4f8d37471472aabae169bddc1231c', 'request_id': 'req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.829305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c4f8d37471472aabae169bddc1231c', 'request_id': 'req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.840862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] The token cache is being invalidate because project 25c4f8d37471472aabae169bddc1231c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:18.841262 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.842082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:18.844010 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1319/2634] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:18 2022] DELETE /identity/v3/projects/25c4f8d37471472aabae169bddc1231c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:18.847802 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.849271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.866572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.867217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.867540 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.868329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.869613 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.871470 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.905211 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1316/2635] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:18.909296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:18.910961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:18.927606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c273a-3571-4f63-9ba6-91b5f0c79fe3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.942534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:18.989705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9005b-925d-4fe2-9fd5-dedac20a612c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:18.994445 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.995060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.995298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:18.995427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.995644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.995928 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:18.996204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:18.996444 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.996584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] PATH_INFO: `/v3/users/78753964646b40089043d3562af39e4d` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:18.997620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78753964646b40089043d3562af39e4d)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:18.998403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:18.999541 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.026421 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1317/2636] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:18 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:19.030920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.042990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454f390-0d13-465b-869d-cf2032ca5b3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.048028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.048671 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.048947 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.049215 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.049861 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.051495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.062000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78753964646b40089043d3562af39e4d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.063367 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1318/2637] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:19.068539 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.071670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78753964646b40089043d3562af39e4d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.079128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e2698-0762-400d-a69a-b5e570ea8b04 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.081173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78753964646b40089043d3562af39e4d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.081840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78753964646b40089043d3562af39e4d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.084506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.085303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.085572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.085917 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] PATH_INFO: `/v3/projects/8ff478ee0f3f47bead6f3caf1c8740a1/users/84359f792c374f819d4dcfa572b4f7a7/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.095056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78753964646b40089043d3562af39e4d'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.101829 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff478ee0f3f47bead6f3caf1c8740a1, user_id=84359f792c374f819d4dcfa572b4f7a7, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.105201 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1320/2638] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:18 2022] DELETE /identity/v3/users/78753964646b40089043d3562af39e4d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:19.105330 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.125153 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1319/2639] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:19 2022] PUT /identity/v3/projects/8ff478ee0f3f47bead6f3caf1c8740a1/users/84359f792c374f819d4dcfa572b4f7a7/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:19.129131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.130708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.211968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042d1bc-a984-4391-afbc-2e4fa7a6c936 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.258531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:19.280021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfae28-85e8-458c-aee2-4cdedd00fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.280655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfae28-85e8-458c-aee2-4cdedd00fdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.280950 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfae28-85e8-458c-aee2-4cdedd00fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.281227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfae28-85e8-458c-aee2-4cdedd00fdd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.281866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfae28-85e8-458c-aee2-4cdedd00fdd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.283512 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfae28-85e8-458c-aee2-4cdedd00fdd3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.294810 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1321/2640] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:19.299081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.310225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae33e80-7a48-491f-acd8-ab3fbd326405 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.315748 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.316422 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.316719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.317001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] PATH_INFO: `/v3/projects/8ff478ee0f3f47bead6f3caf1c8740a1/users/84359f792c374f819d4dcfa572b4f7a7/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.318996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff478ee0f3f47bead6f3caf1c8740a1, user_id=84359f792c374f819d4dcfa572b4f7a7, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.321684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.333951 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1320/2641] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:19 2022] PUT /identity/v3/projects/8ff478ee0f3f47bead6f3caf1c8740a1/users/84359f792c374f819d4dcfa572b4f7a7/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:19.345515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.345867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.346141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.376897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] MFA Rules not processed for user `84359f792c374f819d4dcfa572b4f7a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:19.378434 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.395214 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.396918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.420544 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:19.466676 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1322/2642] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:19.470590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e65ee0-dff3-4885-a5c8-e39161407cc0 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.478002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.478642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.478910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.479171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] PATH_INFO: `/v3/projects/a39b66f716c0419ca1b36c272797f3a6` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.479492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.480319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39b66f716c0419ca1b36c272797f3a6)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.480850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.482234 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.534864 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39b66f716c0419ca1b36c272797f3a6', 'request_id': 'req-dd2c5de9-0295-4fbb-8136-7fd67f001966'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.543708 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee4251-92af-47d0-9fd9-d422fd0e370f None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.543967 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39b66f716c0419ca1b36c272797f3a6', 'request_id': 'req-dd2c5de9-0295-4fbb-8136-7fd67f001966'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.549892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39b66f716c0419ca1b36c272797f3a6', 'request_id': 'req-dd2c5de9-0295-4fbb-8136-7fd67f001966'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.560929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] The token cache is being invalidate because project a39b66f716c0419ca1b36c272797f3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:19.561190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.561823 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-dd2c5de9-0295-4fbb-8136-7fd67f001966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:19.563357 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1321/2643] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:19 2022] DELETE /identity/v3/projects/a39b66f716c0419ca1b36c272797f3a6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:19.570411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906e75e-3da1-4977-84d3-965bf1e18811 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.570692 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906e75e-3da1-4977-84d3-965bf1e18811 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.570938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906e75e-3da1-4977-84d3-965bf1e18811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.594294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d906e75e-3da1-4977-84d3-965bf1e18811 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:19.595530 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d906e75e-3da1-4977-84d3-965bf1e18811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.599853 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.600675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.600934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.601199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.602958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.639819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:19.658488 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1322/2644] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:19.666761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.667196 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.667586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.671439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.672965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.682816 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1323/2645] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:19.689313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:19.690520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:19.700427 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1323/2646] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:19.705021 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.715412 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecee66-bc29-4aa1-9cdf-80dc1e42d6d8 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.725148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.725761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.726032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.726297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.726972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:19.727304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.728914 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.745233 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1324/2647] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:19.746368 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.755731 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84b6cb-d4be-44a7-8357-11aaff5d370e None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.760329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.760928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.761195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.761456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.762070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.764135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.794828 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1324/2648] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:19 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:19.798972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.809487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28566ed-e62f-49cd-9d8b-49ad8aa11ecd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.866306 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.867081 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.867390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.867771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.868496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.870517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.895336 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1325/2649] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:19 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:19.900709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.911788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78139a-f2bb-4e15-b077-c2986e3bf148 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.917770 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.918504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.918819 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.919076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.919807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.921762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.933699 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1325/2650] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:19.938588 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.950601 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde73c5-06b4-4196-9c41-e2c3026ea226 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.956789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.957414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.957669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.957935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:19.958569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:19.960213 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:19.975208 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1326/2651] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:19.976673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:19.992205 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35313711-cb5e-4d68-b0b9-19e270a332a5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:19.996953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1d39b-29f1-4d3b-b74a-524bab084023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:19.997599 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d39b-29f1-4d3b-b74a-524bab084023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:19.997899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d39b-29f1-4d3b-b74a-524bab084023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:19.998144 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d39b-29f1-4d3b-b74a-524bab084023 admin admin] PATH_INFO: `/v3/projects/4b652393210a4597a1f10ca106bceee5/users/2e09624a58864344bbd471b94ff7243d/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.001833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:20.015549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d39b-29f1-4d3b-b74a-524bab084023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b652393210a4597a1f10ca106bceee5, user_id=2e09624a58864344bbd471b94ff7243d, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.018236 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d39b-29f1-4d3b-b74a-524bab084023 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.024057 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.032715 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1326/2652] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:19 2022] PUT /identity/v3/projects/4b652393210a4597a1f10ca106bceee5/users/2e09624a58864344bbd471b94ff7243d/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:20.044366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.044644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.044911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.072757 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] MFA Rules not processed for user `2e09624a58864344bbd471b94ff7243d`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:20.074089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:20.091867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27387a-1652-414b-b35b-93d03558c167 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.134344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:20.139346 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1327/2653] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:20.143811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.161498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84359f792c374f819d4dcfa572b4f7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ff478ee0f3f47bead6f3caf1c8740a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.162461 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.162744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.162999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.163693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:20.164113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.166332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.178446 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1327/2654] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:20.182543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.192720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16b0a3-25f6-4000-88dc-996a958c58de tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.199635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84359f792c374f819d4dcfa572b4f7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ff478ee0f3f47bead6f3caf1c8740a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.200696 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.200984 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.201265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.202211 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.204189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.226588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a99b4-23b2-400e-8340-e9da7217f26d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.231751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.232380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.232652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.232916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.233668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.236721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.237299 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1328/2655] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/projects => generated 353 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:20.241045 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.269834 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1328/2656] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:20.273895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.315719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e1c19-ca4b-4871-8acc-d89b400f1119 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.339854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5b00d-7789-45d0-838b-3d44bc2ee7fe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.347506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.348309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.348654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.349020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.349784 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.352031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.360852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:20.377252 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1329/2657] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:20.380853 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.381902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84359f792c374f819d4dcfa572b4f7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ff478ee0f3f47bead6f3caf1c8740a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.382654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.382992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.383305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.384012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:20.384320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.386023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.392980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0e801e-db7e-4385-af2e-985137395335 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.397741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.398339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.398609 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.398886 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.402438 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1329/2658] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:20.402579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.402579 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.413067 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1330/2659] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:20.417585 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.431506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5841af7-1121-48d5-b980-3f5cb73505ba tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.438478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.439328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.439673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.440168 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.440887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.441349 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.442614 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.452553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc88b-8083-4b09-9271-dc447de595e8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.457394 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1330/2660] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:20.461546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.469600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84359f792c374f819d4dcfa572b4f7a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8ff478ee0f3f47bead6f3caf1c8740a1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.470334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.470526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.470793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] PATH_INFO: `/v3/projects/9d560a777dc34cd7ba535a3051f36c18` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.472241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9d560a777dc34cd7ba535a3051f36c18)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.475321 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.479109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de45d93-e388-415c-bef4-6ef973c609ae admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.486368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7a1db-ff31-4f5d-a522-8df233304f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.487134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7a1db-ff31-4f5d-a522-8df233304f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.487489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7a1db-ff31-4f5d-a522-8df233304f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.487848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7a1db-ff31-4f5d-a522-8df233304f80 admin admin] PATH_INFO: `/v3/projects/b5dbc6fa848347a2af301057c0457cef/users/390e87ac16d24976b37d9c69e4bf3a09/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.510139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7a1db-ff31-4f5d-a522-8df233304f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5dbc6fa848347a2af301057c0457cef, user_id=390e87ac16d24976b37d9c69e4bf3a09, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.519387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7a1db-ff31-4f5d-a522-8df233304f80 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.532448 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1331/2661] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:20 2022] PUT /identity/v3/projects/b5dbc6fa848347a2af301057c0457cef/users/390e87ac16d24976b37d9c69e4bf3a09/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:20.542995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.543294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.543697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.548841 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d560a777dc34cd7ba535a3051f36c18', 'request_id': 'req-e2194c0a-5867-4894-b69f-6c03acbeadf6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.560885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d560a777dc34cd7ba535a3051f36c18', 'request_id': 'req-e2194c0a-5867-4894-b69f-6c03acbeadf6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.567742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d560a777dc34cd7ba535a3051f36c18', 'request_id': 'req-e2194c0a-5867-4894-b69f-6c03acbeadf6'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.574555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] MFA Rules not processed for user `390e87ac16d24976b37d9c69e4bf3a09`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:20.575861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:20.580625 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] The token cache is being invalidate because project 9d560a777dc34cd7ba535a3051f36c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:20.581248 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.581505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.583493 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1331/2662] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:20 2022] DELETE /identity/v3/projects/9d560a777dc34cd7ba535a3051f36c18 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:20.589599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:20.647201 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1332/2663] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:20.652360 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.653810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:20.726110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2194c0a-5867-4894-b69f-6c03acbeadf6 tempest-VolumeQuotaClassesTest-1075747977 tempest-VolumeQuotaClassesTest-1075747977-project-admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.731298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.731914 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.732189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.732460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.733097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.735149 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.763903 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1332/2664] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:20.768386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.781141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3e831-469c-4c6b-b128-60c5201acfc6 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.794145 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.796047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:20.841155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.841992 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.842114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.842283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.842893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.844819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.847532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:20.865485 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd6c80-6379-4813-b558-d7886d5bc3e5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.868285 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1333/2665] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:20.872302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.873256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.873900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.874180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.874449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] PATH_INFO: `/v3/users/14241cfb17ab42b1a027c1f436ad1ada` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.875728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14241cfb17ab42b1a027c1f436ad1ada)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.877733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.882126 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd99b40-592c-4ef2-aea5-f0f04707e414 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.888226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.888404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.888673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.888948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.890062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.891971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.907203 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1334/2666] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:20.908368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.919348 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82451b96-5663-4dec-aa6e-e2acfc9b2d70 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:20.923972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:20.924703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.924980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.925268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] PATH_INFO: `/v3/projects/4e8faeea741f463ab7f2cf537ec81bf4/users/c5b69d8f6e344d9b9c537d7235d59766/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.929574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14241cfb17ab42b1a027c1f436ad1ada'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.937435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14241cfb17ab42b1a027c1f436ad1ada'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.940789 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8faeea741f463ab7f2cf537ec81bf4, user_id=c5b69d8f6e344d9b9c537d7235d59766, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:20.943405 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:20.950675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14241cfb17ab42b1a027c1f436ad1ada'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.951581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14241cfb17ab42b1a027c1f436ad1ada'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.957273 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1335/2667] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:20 2022] PUT /identity/v3/projects/4e8faeea741f463ab7f2cf537ec81bf4/users/c5b69d8f6e344d9b9c537d7235d59766/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:20.958405 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-5713a844-b6d9-4f94-af64-6ec98d23b807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14241cfb17ab42b1a027c1f436ad1ada'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:20.961313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:20.962919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:20.965381 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1333/2668] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:20 2022] DELETE /identity/v3/users/14241cfb17ab42b1a027c1f436ad1ada => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:20.969538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:20.969811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:20.970076 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:20.989637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:20.990883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:21.028172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b7ca5-cda8-426f-8d8a-a715edd27fca admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:21.036921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:21.051575 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1334/2669] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:21.061718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:21.063187 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:21.088066 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:21.088676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.088963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.089216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.089848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:21.091542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:21.101366 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1336/2670] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:21.105560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:21.115270 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b57a16-d8eb-4f4e-b545-18c1e4589ea3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:21.119754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c26ff6-ca92-4b23-b195-0f9e10c1f09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:21.120382 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26ff6-ca92-4b23-b195-0f9e10c1f09e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.120520 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bab92e-04fd-42eb-b563-3b5630d233ad None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:21.120661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26ff6-ca92-4b23-b195-0f9e10c1f09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.120906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c26ff6-ca92-4b23-b195-0f9e10c1f09e admin admin] PATH_INFO: `/v3/projects/4e8faeea741f463ab7f2cf537ec81bf4/users/c5b69d8f6e344d9b9c537d7235d59766/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.122852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26ff6-ca92-4b23-b195-0f9e10c1f09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8faeea741f463ab7f2cf537ec81bf4, user_id=c5b69d8f6e344d9b9c537d7235d59766, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:21.124257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:21.124867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.125140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.125407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.125504 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c26ff6-ca92-4b23-b195-0f9e10c1f09e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:21.134982 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:21.135517 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1337/2671] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:21 2022] PUT /identity/v3/projects/4e8faeea741f463ab7f2cf537ec81bf4/users/c5b69d8f6e344d9b9c537d7235d59766/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:21.140289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:21.147985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.148267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.148542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.151750 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1335/2672] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:21.179458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] MFA Rules not processed for user `c5b69d8f6e344d9b9c537d7235d59766`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:21.180684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:21.257239 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1338/2673] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:21.346097 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:21.347695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:21.359370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:21.409078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554af04a-49a2-4698-8297-865402f31d09 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:21.436798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:21.438264 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:21.461152 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:21.461777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.462052 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.462319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.464475 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:21.521798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd203e9-3524-4bca-8b3c-1258fac485a9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:21.528849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:21.529533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:21.530178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.530317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:21.530587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.530932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] PATH_INFO: `/v3/projects/8732fd52697240218cc40b11accb34e3` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.532216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8732fd52697240218cc40b11accb34e3)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:21.534172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:21.540642 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1336/2674] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:21.561772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:21.592420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8732fd52697240218cc40b11accb34e3', 'request_id': 'req-c6270980-adb9-4a17-b821-54564ad824d1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.602537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8732fd52697240218cc40b11accb34e3', 'request_id': 'req-c6270980-adb9-4a17-b821-54564ad824d1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.611488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8732fd52697240218cc40b11accb34e3', 'request_id': 'req-c6270980-adb9-4a17-b821-54564ad824d1'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.625266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] The token cache is being invalidate because project 8732fd52697240218cc40b11accb34e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:21.625628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.626391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.628192 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1339/2675] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:21 2022] DELETE /identity/v3/projects/8732fd52697240218cc40b11accb34e3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:21.632091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:21.645458 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e17071-aea4-4b6b-9ee0-f081900b7df5 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:21.733395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:21.734175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:21.734516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:21.735053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] PATH_INFO: `/v3/users/84359f792c374f819d4dcfa572b4f7a7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:21.736176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] RBAC: Authorizing `identity:delete_user(user_id=84359f792c374f819d4dcfa572b4f7a7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:21.738151 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:21.762367 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:21.804994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84359f792c374f819d4dcfa572b4f7a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.813632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84359f792c374f819d4dcfa572b4f7a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.836542 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84359f792c374f819d4dcfa572b4f7a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.837348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84359f792c374f819d4dcfa572b4f7a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.845084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c8889cf6-4aa1-42d3-bc4f-1ebe256eaafe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84359f792c374f819d4dcfa572b4f7a7'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:21.851880 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1337/2676] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:21 2022] DELETE /identity/v3/users/84359f792c374f819d4dcfa572b4f7a7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:22.157613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.159576 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:22.176160 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6270980-adb9-4a17-b821-54564ad824d1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.237243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.237907 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.238175 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.238443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] PATH_INFO: `/v3/projects/8ff478ee0f3f47bead6f3caf1c8740a1` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.239699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff478ee0f3f47bead6f3caf1c8740a1)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.241656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.298592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff478ee0f3f47bead6f3caf1c8740a1', 'request_id': 'req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:22.307018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff478ee0f3f47bead6f3caf1c8740a1', 'request_id': 'req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:22.314388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff478ee0f3f47bead6f3caf1c8740a1', 'request_id': 'req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:22.325573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] The token cache is being invalidate because project 8ff478ee0f3f47bead6f3caf1c8740a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:22.325858 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:22.326507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-4cef55e8-d0a0-4570-8bb0-20f2063f7efa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:22.328293 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1340/2677] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:22 2022] DELETE /identity/v3/projects/8ff478ee0f3f47bead6f3caf1c8740a1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:22.334262 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.334827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.334935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.360719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:22.362114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:22.424454 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1338/2678] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:22.432652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.432953 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.433279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.454217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:22.455479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:22.464096 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1341/2679] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:22.468565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.478824 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83574-b01f-4d6b-8daf-3e2599b29953 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.484421 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.485018 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.485285 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.485688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.486497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:22.486793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.488449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.499947 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1339/2680] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:22.504409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.514050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3519d-a61b-403d-9d82-3adf97c9bd6a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.519624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.520242 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.520510 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.520772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.521404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.523396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.552534 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1342/2681] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:22 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:22.556747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.566870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97be13c-520e-4370-b765-2bfe7130dbb7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.628565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.629390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.629514 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.629708 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.630338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.632314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.656611 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1340/2682] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:22 2022] POST /identity/v3/users => generated 280 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:22.662583 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.676590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775051f-7935-4f79-b1b2-89a64c434206 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.684595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.685226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.685523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.685779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.686408 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.688032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.699963 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1343/2683] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:22.704025 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.714517 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986466a-5193-4265-9699-922f1fd09e9f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.719966 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b0180-cfd9-4dcd-a228-414856a0ec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.720560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0180-cfd9-4dcd-a228-414856a0ec81 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.720832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0180-cfd9-4dcd-a228-414856a0ec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.721164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b0180-cfd9-4dcd-a228-414856a0ec81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.721810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b0180-cfd9-4dcd-a228-414856a0ec81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.724260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b0180-cfd9-4dcd-a228-414856a0ec81 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.735938 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1341/2684] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:22.740296 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:22.745546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:22.752892 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce78ad-ffe6-4137-a347-0dd9eb601767 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:22.759758 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:22.760455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.760728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.760993 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] PATH_INFO: `/v3/projects/1b98b2286e6e4268b2c9227054999358/users/ef7dc9e046704b16bc7c66176b167927/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.776567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b98b2286e6e4268b2c9227054999358, user_id=ef7dc9e046704b16bc7c66176b167927, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:22.779364 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:22.797184 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1344/2685] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:22 2022] PUT /identity/v3/projects/1b98b2286e6e4268b2c9227054999358/users/ef7dc9e046704b16bc7c66176b167927/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:22.809642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:22.809971 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:22.810239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:22.818742 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:22.841207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] MFA Rules not processed for user `ef7dc9e046704b16bc7c66176b167927`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:22.842447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:22.925803 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1342/2686] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:23.017656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:23.019636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.080091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48601d85-a573-43bf-8312-2cdd26e48e53 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:23.083715 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:23.084350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.084623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.084891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.093584 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:23.095008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:23.110513 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1345/2687] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:23.132004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:23.239792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029cf30e-e1d5-4c25-ba84-ad5dfbd63063 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:23.241416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:23.246380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:23.247129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.247447 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.247720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] PATH_INFO: `/v3/users/ef7dc9e046704b16bc7c66176b167927` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.249186 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7dc9e046704b16bc7c66176b167927)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:23.250542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c42ff-d8db-44a9-98c3-a1996bc3682e service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:23.251100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:23.254556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:23.255399 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.255760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.256028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.257852 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.301648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7dc9e046704b16bc7c66176b167927'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.309456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7dc9e046704b16bc7c66176b167927'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.320600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:23.322050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d84c6d-e505-4c3a-a022-5b24bc17c480 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:23.322195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7dc9e046704b16bc7c66176b167927'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.322779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7dc9e046704b16bc7c66176b167927'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.329929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7dc9e046704b16bc7c66176b167927'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.331462 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1346/2688] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:23.335836 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1343/2689] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:23 2022] DELETE /identity/v3/users/ef7dc9e046704b16bc7c66176b167927 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:23.340390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.340668 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.340938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.360507 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:23.361724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.421752 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1347/2690] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:23.432049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:23.433686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.450259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:23.497290 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e19f1b-198b-49aa-b80e-5aebd7d70a0c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:23.500897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74074f33-844b-4f51-aac4-39650f290508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:23.501499 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74074f33-844b-4f51-aac4-39650f290508 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.501768 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74074f33-844b-4f51-aac4-39650f290508 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.502031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74074f33-844b-4f51-aac4-39650f290508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.510885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74074f33-844b-4f51-aac4-39650f290508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:23.512371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74074f33-844b-4f51-aac4-39650f290508 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:23.521816 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1344/2691] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:23.804560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:23.806651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.820125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e283e-be6b-4d3b-adbd-79142c32b85a None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:23.825973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:23.826656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.827028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.827414 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] PATH_INFO: `/v3/projects/1b98b2286e6e4268b2c9227054999358` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.828816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b98b2286e6e4268b2c9227054999358)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:23.831230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:23.850060 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:23.883634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b98b2286e6e4268b2c9227054999358', 'request_id': 'req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.893058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b98b2286e6e4268b2c9227054999358', 'request_id': 'req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.899634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b98b2286e6e4268b2c9227054999358', 'request_id': 'req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.910943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] The token cache is being invalidate because project 1b98b2286e6e4268b2c9227054999358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:23.911261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.912001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:23.913953 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1348/2692] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:23 2022] DELETE /identity/v3/projects/1b98b2286e6e4268b2c9227054999358 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:23.919085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e805505-113f-469e-acf0-c708e231ab11 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.919486 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e805505-113f-469e-acf0-c708e231ab11 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.919820 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e805505-113f-469e-acf0-c708e231ab11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.927833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:23.930321 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.939735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-7e805505-113f-469e-acf0-c708e231ab11 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:23.940921 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-7e805505-113f-469e-acf0-c708e231ab11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:23.987728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9d45d-c5df-4fed-9a6a-a55277cfc10b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:23.991651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:23.992265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:23.992538 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:23.992805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:23.994574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:24.000728 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1345/2693] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:24.009327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.009607 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.009869 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.029245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:24.030418 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:24.039416 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1346/2694] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:24.045137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.054409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb483ef5-69bb-4bfd-be0b-bcb0e143ccc2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.059162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.059789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.060061 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.060327 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.060972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:24.061270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.062870 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.067608 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.069053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.075216 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1347/2695] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:24.079376 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1349/2696] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:24.079923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.090212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f2f5-302a-4bde-8458-740a2560dfd8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.095412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.096002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.096274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.096538 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.097157 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.099163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.132249 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1348/2697] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:24 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:24.132600 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.144028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f29aed-f3ce-4e42-86b0-173e912953cb service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.196138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:24.206832 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.207500 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.207769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.208041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.208654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.210552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.232897 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1350/2698] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:24 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:24.237341 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.247793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98d50d-061f-4b92-9d7a-2def42e11d4c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.254508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.255119 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.255431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.255696 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.256339 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.257936 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.271245 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1349/2699] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:24.273691 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.284996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513f3c6-8a61-45f4-94d7-223056e99966 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.290652 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a38a95-905d-4aa5-922a-f0cb1b88b949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.291407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38a95-905d-4aa5-922a-f0cb1b88b949 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.291780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38a95-905d-4aa5-922a-f0cb1b88b949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.292176 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a38a95-905d-4aa5-922a-f0cb1b88b949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.292954 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38a95-905d-4aa5-922a-f0cb1b88b949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.294648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a38a95-905d-4aa5-922a-f0cb1b88b949 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.305403 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1351/2700] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:24.309846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.320507 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513557e7-a940-4a6c-9c7c-aa9c11fd886e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.326892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.327586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.327867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.328136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] PATH_INFO: `/v3/projects/7601b75add294ad0af090dfbe639e5fb/users/98ed8708b2f145998d45f8d1ae45bd89/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.347678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7601b75add294ad0af090dfbe639e5fb, user_id=98ed8708b2f145998d45f8d1ae45bd89, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.350751 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.365127 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1350/2701] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:24 2022] PUT /identity/v3/projects/7601b75add294ad0af090dfbe639e5fb/users/98ed8708b2f145998d45f8d1ae45bd89/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:24.375727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.376020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.376291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.410639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] MFA Rules not processed for user `98ed8708b2f145998d45f8d1ae45bd89`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:24.411980 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:24.485047 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1352/2702] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:24.497923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:24.499901 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:24.550048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:24.568381 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355274e-a524-41a6-b705-7413390e1d34 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:24.572354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8388bedc588f4f5f94f1e732d46e83b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:24.572992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:24.573258 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:24.573526 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:24.583392 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:24.584872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:24.595084 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1351/2703] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:27.774032 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:27.775630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:27.862313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006831ee-bb0a-4f3a-8666-404f7328bd03 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:27.869824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:27.870579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:27.870915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:27.871273 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:27.873676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:27.886318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:27.888266 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5252c706-ac8e-4673-aa1c-befe1204f562 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:27.898636 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1353/2704] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:28.061749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:28.063441 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:28.146231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe82d0-b4fa-4269-940c-1de44a2482a9 service glance] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:28.151274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:28.152091 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:28.152370 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:28.152649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] PATH_INFO: `/v3/users/2e09624a58864344bbd471b94ff7243d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:28.153788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e09624a58864344bbd471b94ff7243d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:28.155743 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:28.216209 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e09624a58864344bbd471b94ff7243d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.228035 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e09624a58864344bbd471b94ff7243d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.245546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e09624a58864344bbd471b94ff7243d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.245906 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e09624a58864344bbd471b94ff7243d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.264645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e09624a58864344bbd471b94ff7243d'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.270934 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1352/2705] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:28 2022] DELETE /identity/v3/users/2e09624a58864344bbd471b94ff7243d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:28.276110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:28.276620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:28.276620 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:28.304238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:28.305470 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:28.355483 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:28.388431 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1354/2706] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:28.398219 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:28.399891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:28.408765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:28.410487 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:28.423363 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0aebac-3c4a-4088-ae4a-f0347753544b None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:28.428051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:28.428686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:28.428955 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:28.429221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] PATH_INFO: `/v3/users/26e7fce78cc74e0aace9b0956762c126` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:28.430329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e7fce78cc74e0aace9b0956762c126)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:28.432297 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:28.462617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab3a4a-9fd6-4c54-aea1-fefc0c9eb7e8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:28.466309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:28.466916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:28.467202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:28.467468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:28.476652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:28.478179 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:28.484506 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e7fce78cc74e0aace9b0956762c126'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.487583 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1353/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:28.492603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e7fce78cc74e0aace9b0956762c126'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.500567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e7fce78cc74e0aace9b0956762c126'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.501457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e7fce78cc74e0aace9b0956762c126'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.513985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e7fce78cc74e0aace9b0956762c126'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.519950 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1355/2708] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:28 2022] DELETE /identity/v3/users/26e7fce78cc74e0aace9b0956762c126 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:28.764899 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:28.771337 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:28.775481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725e00d-b7fe-4215-b054-416843c7ea90 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:28.831542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:28.836939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:28.851676 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:28.852401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:28.852745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:28.853086 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] PATH_INFO: `/v3/projects/6b1f92f790a043bf9848695a5c517857` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:28.854301 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1f92f790a043bf9848695a5c517857)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:28.856439 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:28.910619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ada64-90f6-4201-af97-8257e83544bf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:28.911501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1f92f790a043bf9848695a5c517857', 'request_id': 'req-f734f0f3-b771-4a11-b3f4-84bf538178fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.920111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1f92f790a043bf9848695a5c517857', 'request_id': 'req-f734f0f3-b771-4a11-b3f4-84bf538178fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.926776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1f92f790a043bf9848695a5c517857', 'request_id': 'req-f734f0f3-b771-4a11-b3f4-84bf538178fa'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.939718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] The token cache is being invalidate because project 6b1f92f790a043bf9848695a5c517857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:28.939997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.940774 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:28.942324 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1354/2709] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:28 2022] DELETE /identity/v3/projects/6b1f92f790a043bf9848695a5c517857 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:28.946271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:28.947616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:28.972283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:28.972942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:28.973192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:28.973781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] PATH_INFO: `/v3/projects/4b652393210a4597a1f10ca106bceee5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:28.974596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b652393210a4597a1f10ca106bceee5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:28.976546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:28.988244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.016198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734f0f3-b771-4a11-b3f4-84bf538178fa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.027373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b652393210a4597a1f10ca106bceee5', 'request_id': 'req-c934f84b-29fd-4120-a170-78e85d97f0dd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.035920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b652393210a4597a1f10ca106bceee5', 'request_id': 'req-c934f84b-29fd-4120-a170-78e85d97f0dd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.042703 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b652393210a4597a1f10ca106bceee5', 'request_id': 'req-c934f84b-29fd-4120-a170-78e85d97f0dd'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.054535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] The token cache is being invalidate because project 4b652393210a4597a1f10ca106bceee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:29.054871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.056515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.056673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.060731 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1356/2710] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:28 2022] DELETE /identity/v3/projects/4b652393210a4597a1f10ca106bceee5 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:29.065818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.067619 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.077457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.078822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.079226 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.079453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] PATH_INFO: `/v3/users/6861061d30fb4f13b389116d73653400` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.080888 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6861061d30fb4f13b389116d73653400)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.082939 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.135374 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6861061d30fb4f13b389116d73653400'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.136478 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f84b-29fd-4120-a170-78e85d97f0dd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.142433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.143235 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.143732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.144013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] PATH_INFO: `/v3/users/390e87ac16d24976b37d9c69e4bf3a09` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.144293 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6861061d30fb4f13b389116d73653400'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.145462 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] RBAC: Authorizing `identity:delete_user(user_id=390e87ac16d24976b37d9c69e4bf3a09)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.147624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.164494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6861061d30fb4f13b389116d73653400'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.165697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6861061d30fb4f13b389116d73653400'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.172639 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6861061d30fb4f13b389116d73653400'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.179906 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1355/2711] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:28 2022] DELETE /identity/v3/users/6861061d30fb4f13b389116d73653400 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:29.199346 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390e87ac16d24976b37d9c69e4bf3a09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.208052 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390e87ac16d24976b37d9c69e4bf3a09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.215369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390e87ac16d24976b37d9c69e4bf3a09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.216056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390e87ac16d24976b37d9c69e4bf3a09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.232590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390e87ac16d24976b37d9c69e4bf3a09'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.238999 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1357/2712] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:29 2022] DELETE /identity/v3/users/390e87ac16d24976b37d9c69e4bf3a09 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:29.374921 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.377552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.465286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc9d6-8630-42fc-bf12-8398288069e4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.525612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.526260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.526521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.526789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] PATH_INFO: `/v3/projects/0c6cf8012e2f454ab56fcd57289c4449` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.528106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c6cf8012e2f454ab56fcd57289c4449)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.530423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.530649 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.531989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.550351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.555361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fb51e-8f80-4dc4-a56b-65adeb20517f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.560651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.561293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.561570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.568693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] PATH_INFO: `/v3/projects/b5dbc6fa848347a2af301057c0457cef` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.569974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5dbc6fa848347a2af301057c0457cef)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.571963 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.588704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.591348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c6cf8012e2f454ab56fcd57289c4449', 'request_id': 'req-2b17bec3-63fe-4f42-9373-9886932f6dce'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.603156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c6cf8012e2f454ab56fcd57289c4449', 'request_id': 'req-2b17bec3-63fe-4f42-9373-9886932f6dce'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.610586 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c6cf8012e2f454ab56fcd57289c4449', 'request_id': 'req-2b17bec3-63fe-4f42-9373-9886932f6dce'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.621833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] The token cache is being invalidate because project 0c6cf8012e2f454ab56fcd57289c4449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:29.622110 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.622741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b17bec3-63fe-4f42-9373-9886932f6dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.624335 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1356/2713] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:29 2022] DELETE /identity/v3/projects/0c6cf8012e2f454ab56fcd57289c4449 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:29.626124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5dbc6fa848347a2af301057c0457cef', 'request_id': 'req-b603c4df-64b2-4d7e-b7dd-d27780ea772b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.629329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9bd2-039d-4d95-ad61-4f09bc0262d6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.629792 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9bd2-039d-4d95-ad61-4f09bc0262d6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.630063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae9bd2-039d-4d95-ad61-4f09bc0262d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.634903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5dbc6fa848347a2af301057c0457cef', 'request_id': 'req-b603c4df-64b2-4d7e-b7dd-d27780ea772b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.641681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5dbc6fa848347a2af301057c0457cef', 'request_id': 'req-b603c4df-64b2-4d7e-b7dd-d27780ea772b'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.651168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-52ae9bd2-039d-4d95-ad61-4f09bc0262d6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:29.652540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] The token cache is being invalidate because project b5dbc6fa848347a2af301057c0457cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:29.652848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-52ae9bd2-039d-4d95-ad61-4f09bc0262d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.652960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.653605 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.658324 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1358/2714] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:29 2022] DELETE /identity/v3/projects/b5dbc6fa848347a2af301057c0457cef => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:29.659530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.660908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.721155 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1357/2715] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:29.729617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.729916 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.730159 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.730995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603c4df-64b2-4d7e-b7dd-d27780ea772b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.735632 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.736261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.736533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.736798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] PATH_INFO: `/v3/users/c5b69d8f6e344d9b9c537d7235d59766` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.737930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5b69d8f6e344d9b9c537d7235d59766)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.739875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.749455 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:29.750648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.758770 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1358/2716] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:29.763300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.773812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6dbc09-69ed-4f53-be6e-9982d76f4e50 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.779600 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.780369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.780810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.781180 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.782036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:29.782713 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.784572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.796214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5b69d8f6e344d9b9c537d7235d59766'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.801313 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1359/2717] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:29.802401 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.805062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5b69d8f6e344d9b9c537d7235d59766'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.812247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5b69d8f6e344d9b9c537d7235d59766'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.812848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc505f-cc2c-4c75-828f-f4615af07086 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:29.812998 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5b69d8f6e344d9b9c537d7235d59766'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.817871 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:29.818457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:29.818726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:29.818983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:29.819659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:29.821559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:29.827293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5b69d8f6e344d9b9c537d7235d59766'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:29.834075 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1359/2718] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:29 2022] DELETE /identity/v3/users/c5b69d8f6e344d9b9c537d7235d59766 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:29.854477 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1360/2719] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:29 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:29.858687 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:29.860308 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:29.890656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:29.935492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b08e9-7c6b-4713-95c4-b9bca02c2df0 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.003499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.004594 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.004877 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.005141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.005773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.007719 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.033711 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1360/2720] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:29 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:30.035065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.046863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437bd813-894f-4481-b972-8ac28f4e2d98 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.051996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.052702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.053308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.053308 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.054095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.055817 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.066842 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1361/2721] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:30.071125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.084345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c18a7-4bd7-400a-a4c1-092b8a436dc5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.089288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.089902 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.090243 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.090542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.091350 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.092982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.104225 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1361/2722] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:30.108754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.119624 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a1b57-eee3-4e24-9dbe-73a6b860a4d6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.125745 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.127704 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:30.130174 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de511490-4af1-4c7e-801c-81e81ae490bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.130856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de511490-4af1-4c7e-801c-81e81ae490bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.131127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de511490-4af1-4c7e-801c-81e81ae490bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.131428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de511490-4af1-4c7e-801c-81e81ae490bb admin admin] PATH_INFO: `/v3/projects/4db57650d7714915b6dfde6ff38ade3c/users/d2f8042166b14816bfba5f58761f4368/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.148591 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ad8f5-4d72-4333-9677-40519ee7868c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.151361 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de511490-4af1-4c7e-801c-81e81ae490bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db57650d7714915b6dfde6ff38ade3c, user_id=d2f8042166b14816bfba5f58761f4368, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.154233 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de511490-4af1-4c7e-801c-81e81ae490bb admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.156678 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.159951 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.160226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.160556 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] PATH_INFO: `/v3/projects/4e8faeea741f463ab7f2cf537ec81bf4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.162397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8faeea741f463ab7f2cf537ec81bf4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.164517 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.176495 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1362/2723] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:30 2022] PUT /identity/v3/projects/4db57650d7714915b6dfde6ff38ade3c/users/d2f8042166b14816bfba5f58761f4368/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:30.192572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.192828 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.193106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.225925 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] MFA Rules not processed for user `d2f8042166b14816bfba5f58761f4368`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:30.227358 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:30.229287 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8faeea741f463ab7f2cf537ec81bf4', 'request_id': 'req-f30fe7d7-fde6-408a-8712-0aaee2234a8c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.238767 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8faeea741f463ab7f2cf537ec81bf4', 'request_id': 'req-f30fe7d7-fde6-408a-8712-0aaee2234a8c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.239320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:30.254349 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8faeea741f463ab7f2cf537ec81bf4', 'request_id': 'req-f30fe7d7-fde6-408a-8712-0aaee2234a8c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.271041 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] The token cache is being invalidate because project 4e8faeea741f463ab7f2cf537ec81bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:30.271425 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.272921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.274785 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1362/2724] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:30 2022] DELETE /identity/v3/projects/4e8faeea741f463ab7f2cf537ec81bf4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:30.315268 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1363/2725] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:30.332390 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.333961 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:30.405489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:30.435640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fe7d7-fde6-408a-8712-0aaee2234a8c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.440520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db532850-fe17-45e5-9633-3cd44a86bf6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.441317 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532850-fe17-45e5-9633-3cd44a86bf6c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.441680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532850-fe17-45e5-9633-3cd44a86bf6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.442056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532850-fe17-45e5-9633-3cd44a86bf6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.452269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db532850-fe17-45e5-9633-3cd44a86bf6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.453887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db532850-fe17-45e5-9633-3cd44a86bf6c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.467662 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1363/2726] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:30.493072 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.494487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:30.568089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cede6-0fc4-40c4-a868-69b19efc4b64 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.573243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.573893 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.574166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.574463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] PATH_INFO: `/v3/users/d2f8042166b14816bfba5f58761f4368` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.575725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f8042166b14816bfba5f58761f4368)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.577622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.627644 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f8042166b14816bfba5f58761f4368'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.636660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f8042166b14816bfba5f58761f4368'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.649989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f8042166b14816bfba5f58761f4368'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.650684 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f8042166b14816bfba5f58761f4368'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.657900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f8042166b14816bfba5f58761f4368'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:30.664170 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1364/2727] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:30 2022] DELETE /identity/v3/users/d2f8042166b14816bfba5f58761f4368 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:30.668977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.669260 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.669529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.690623 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:30.691978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:30.761730 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1364/2728] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:30.771758 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:30.773304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:30.810938 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:30.841956 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ae71e-3cab-4b2d-b765-27d31f3e89f8 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:30.847411 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad778887-cbfa-4fd1-8e54-1ceabe66cab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:30.848049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778887-cbfa-4fd1-8e54-1ceabe66cab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:30.848324 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778887-cbfa-4fd1-8e54-1ceabe66cab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:30.848590 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad778887-cbfa-4fd1-8e54-1ceabe66cab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:30.860380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad778887-cbfa-4fd1-8e54-1ceabe66cab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:30.861821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad778887-cbfa-4fd1-8e54-1ceabe66cab7 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:30.871888 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1365/2729] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:31.187621 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.189439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:31.202438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f347786-5fc2-4377-a0a5-1f57d5be2d9e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.207650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.208480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.208905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.209170 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] PATH_INFO: `/v3/projects/4db57650d7714915b6dfde6ff38ade3c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.210633 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db57650d7714915b6dfde6ff38ade3c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.212630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.264531 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db57650d7714915b6dfde6ff38ade3c', 'request_id': 'req-a36c6b72-d491-49a4-b6bf-1f334c972319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:31.273373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db57650d7714915b6dfde6ff38ade3c', 'request_id': 'req-a36c6b72-d491-49a4-b6bf-1f334c972319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:31.279919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db57650d7714915b6dfde6ff38ade3c', 'request_id': 'req-a36c6b72-d491-49a4-b6bf-1f334c972319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:31.291833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] The token cache is being invalidate because project 4db57650d7714915b6dfde6ff38ade3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:31.292124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:31.292840 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a36c6b72-d491-49a4-b6bf-1f334c972319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:31.294554 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1365/2730] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:31 2022] DELETE /identity/v3/projects/4db57650d7714915b6dfde6ff38ade3c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:31.299740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.300284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.300284 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.320873 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:31.322102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:31.384017 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1366/2731] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:31.392109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.392368 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.392665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.414253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:31.415499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:31.423740 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1366/2732] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:31.428274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.438089 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fcaf2-4c80-4dd7-8a3c-de7d4cf570ea None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.447866 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.448495 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.448788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.449049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.449695 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:31.449987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.451589 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.463365 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1367/2733] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:31.468140 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.478459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1f53b-e38e-4331-957e-41f322b5b13d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.484312 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.484947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.485219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.485486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.486166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.488194 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.523712 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1367/2734] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:31 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:31.528254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.539669 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab827e10-0dce-4b61-8c64-6b729e52ec69 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.601924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.602545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.602822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.603093 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.603766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.605756 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.627512 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1368/2735] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:31 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:31.631999 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.642986 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24dea0-e74a-42fd-a890-4223cc9d83d1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.648231 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.648940 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.649310 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.649670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.650334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.652028 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.663785 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1368/2736] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:31.668077 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.678942 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca642dcf-1122-4f9c-a73f-b12672f84e7f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.684202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d5883-e35a-471f-ab2f-aa2bf3ed547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.684834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d5883-e35a-471f-ab2f-aa2bf3ed547f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.685235 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d5883-e35a-471f-ab2f-aa2bf3ed547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.685508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d5883-e35a-471f-ab2f-aa2bf3ed547f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.686133 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d5883-e35a-471f-ab2f-aa2bf3ed547f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.687762 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d5883-e35a-471f-ab2f-aa2bf3ed547f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.700366 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1369/2737] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:31.704867 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.715826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622857ae-8754-4e10-bb5b-997f06c56cc2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.720896 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.721546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.721825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.722099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] PATH_INFO: `/v3/projects/9565230aa1b74f769936233fde3a5eb7/users/3bbcbadccb8c4876aa34f7737057796c/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.738432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9565230aa1b74f769936233fde3a5eb7, user_id=3bbcbadccb8c4876aa34f7737057796c, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.741200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.743407 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:31.758460 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1369/2738] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:31 2022] PUT /identity/v3/projects/9565230aa1b74f769936233fde3a5eb7/users/3bbcbadccb8c4876aa34f7737057796c/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:31.770705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.770986 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.771287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.800085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] MFA Rules not processed for user `3bbcbadccb8c4876aa34f7737057796c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:31.801204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:31.830345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:31.873361 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1370/2739] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:31.885325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:31.888158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:31.947194 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0744d67-a8bb-4e9d-9995-82ad5ce403cd admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:31.951352 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab40f9b-5d51-435c-8c75-de4d01d49d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:31.951945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab40f9b-5d51-435c-8c75-de4d01d49d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:31.952216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab40f9b-5d51-435c-8c75-de4d01d49d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:31.952552 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab40f9b-5d51-435c-8c75-de4d01d49d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:31.962756 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab40f9b-5d51-435c-8c75-de4d01d49d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:31.964511 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab40f9b-5d51-435c-8c75-de4d01d49d6b service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:31.973696 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1370/2740] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:34.580740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:34.647709 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208b3ea-6ca9-4881-908e-37bfc45d7daa None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:34.652288 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:34.652924 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:34.653203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:34.653473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] PATH_INFO: `/v3/users/3bbcbadccb8c4876aa34f7737057796c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:34.654623 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bbcbadccb8c4876aa34f7737057796c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:34.656667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:34.712313 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bbcbadccb8c4876aa34f7737057796c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:34.720673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bbcbadccb8c4876aa34f7737057796c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:34.737176 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bbcbadccb8c4876aa34f7737057796c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:34.737894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bbcbadccb8c4876aa34f7737057796c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:34.745051 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bbcbadccb8c4876aa34f7737057796c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:34.751236 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1371/2741] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:34 2022] DELETE /identity/v3/users/3bbcbadccb8c4876aa34f7737057796c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:34.756196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:34.756761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:34.756761 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:34.776871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:34.778026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:34.841590 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1371/2742] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:34.851898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:34.853481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:34.912175 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:34.916791 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cda039-51b9-456b-8b89-d3948e6095a5 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:34.921740 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7cf50-7e83-4c2f-b415-14c7af2d4482 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:34.922376 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7cf50-7e83-4c2f-b415-14c7af2d4482 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:34.922689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7cf50-7e83-4c2f-b415-14c7af2d4482 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:34.922963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7cf50-7e83-4c2f-b415-14c7af2d4482 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:34.933297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7cf50-7e83-4c2f-b415-14c7af2d4482 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:34.934754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7cf50-7e83-4c2f-b415-14c7af2d4482 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:34.949676 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1372/2743] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:35.230492 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.231946 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:35.247827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d5321-2821-4c2f-bc4e-e23df073b5d8 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.254267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.255163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.255637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.255922 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] PATH_INFO: `/v3/projects/9565230aa1b74f769936233fde3a5eb7` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.257111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9565230aa1b74f769936233fde3a5eb7)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.259272 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.311419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9565230aa1b74f769936233fde3a5eb7', 'request_id': 'req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:35.320403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9565230aa1b74f769936233fde3a5eb7', 'request_id': 'req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:35.327379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9565230aa1b74f769936233fde3a5eb7', 'request_id': 'req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:35.340197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] The token cache is being invalidate because project 9565230aa1b74f769936233fde3a5eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:35.340475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:35.341206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-b784be79-97c7-49a7-b2f3-388fb0ee4ca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:35.342902 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1372/2744] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:35 2022] DELETE /identity/v3/projects/9565230aa1b74f769936233fde3a5eb7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:35.348162 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.348436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.348702 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.369563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:35.370749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:35.432977 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1373/2745] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:35.441179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.441459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.441730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.465421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:35.466665 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:35.475608 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1373/2746] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:35.480067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.490020 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb96de-cb33-4262-8875-21065dceb9bf None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.494719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.495351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.495617 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.495875 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.496513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:35.496806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.498390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.510271 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1374/2747] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:02:35.515069 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.525542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7543-2346-473d-acf8-02c7f06fc18e None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.530430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.531035 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.531309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.531573 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.532215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.534101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.546648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:35.571794 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1374/2748] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:35 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:35.576417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.588564 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf4dcdc-0817-47cc-8b2a-c996b91c7cbe admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.665801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.666450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.666716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.666977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.667630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.669512 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.693745 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1375/2749] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:35 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:35.698328 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.709004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01713751-e4aa-4b50-a4fd-46823f348794 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.714303 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.714942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.715206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.715522 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.716172 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.717887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.729513 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1375/2750] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:35.734111 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.745117 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90a40e-a95e-4d1a-86b5-9f599cfa7af9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.751080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16f631-acf8-4327-ad57-4aaa08f22cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.751707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16f631-acf8-4327-ad57-4aaa08f22cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.751999 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16f631-acf8-4327-ad57-4aaa08f22cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.752387 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16f631-acf8-4327-ad57-4aaa08f22cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.753037 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16f631-acf8-4327-ad57-4aaa08f22cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.754662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16f631-acf8-4327-ad57-4aaa08f22cf8 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.765824 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1376/2751] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:35.770005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:35.780397 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042094f5-e98d-4e43-9eeb-95e29f904127 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:35.785905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:35.786593 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.787218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.787540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] PATH_INFO: `/v3/projects/eecf50e62b474fa1ae29cf0197bdefe7/users/74cd83f1bed24ac48ea36b91613f021c/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.803406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecf50e62b474fa1ae29cf0197bdefe7, user_id=74cd83f1bed24ac48ea36b91613f021c, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:35.806033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:35.823957 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1376/2752] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:35 2022] PUT /identity/v3/projects/eecf50e62b474fa1ae29cf0197bdefe7/users/74cd83f1bed24ac48ea36b91613f021c/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:35.839033 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c230d3-80db-4292-9008-a855f4e6cb52 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:35.839394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c230d3-80db-4292-9008-a855f4e6cb52 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:35.839659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c230d3-80db-4292-9008-a855f4e6cb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:35.869406 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-36c230d3-80db-4292-9008-a855f4e6cb52 None None] MFA Rules not processed for user `74cd83f1bed24ac48ea36b91613f021c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:35.870601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-36c230d3-80db-4292-9008-a855f4e6cb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:36.042319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:36.064119 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1377/2753] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:36.075627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:36.077404 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:36.137300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f9adf-b479-4b8c-854a-c8694df92d06 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:36.141524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:36.142173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:36.142417 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:36.142684 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:36.152191 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:36.153586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:36.163427 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1377/2754] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:37.533513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:37.534232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:37.534232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:37.564424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] MFA Rules not processed for user `74cd83f1bed24ac48ea36b91613f021c`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:37.565654 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:37.573337 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1378/2755] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:44.017636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:44.019754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:44.098389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e79f72-0eed-40ca-8b2e-3a0b03baef39 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:44.106159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:44.106889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:44.107114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:44.107450 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:44.109551 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:44.185361 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:44.187154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:44.203309 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1378/2756] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:47.468468 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:47.469865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:47.491355 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cff0cb-a8b2-444e-a405-7e21c6939f85 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:47.497240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:47.498116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:47.498614 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:47.499070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] PATH_INFO: `/v3/users/44c365d540894a908b34fe51d854e96e` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:47.500793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c365d540894a908b34fe51d854e96e)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:47.502634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:47.555794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c365d540894a908b34fe51d854e96e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.565054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c365d540894a908b34fe51d854e96e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.582207 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c365d540894a908b34fe51d854e96e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.583011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c365d540894a908b34fe51d854e96e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.590154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-635939ec-a795-41b9-bc1a-5777a44c4730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c365d540894a908b34fe51d854e96e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.596108 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1379/2757] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:47 2022] DELETE /identity/v3/users/44c365d540894a908b34fe51d854e96e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:47.777159 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:47.788185 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef738e6-aeb5-4ce4-be7b-4b3f784923ca service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:47.793286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:47.848444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:47.849007 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:47.849276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:47.849543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] PATH_INFO: `/v3/projects/1bf8c10e733e485f9ac63769727b7151` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:47.850656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf8c10e733e485f9ac63769727b7151)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:47.852592 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:47.901207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf8c10e733e485f9ac63769727b7151', 'request_id': 'req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.909670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf8c10e733e485f9ac63769727b7151', 'request_id': 'req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.917083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf8c10e733e485f9ac63769727b7151', 'request_id': 'req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.928878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] The token cache is being invalidate because project 1bf8c10e733e485f9ac63769727b7151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:47.929232 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.929936 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1d8713c3-5d9d-4150-b459-a3c54ec3ca33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:47.931551 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1379/2758] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:47 2022] DELETE /identity/v3/projects/1bf8c10e733e485f9ac63769727b7151 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:47.937130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:47.937501 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:47.937900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:47.959718 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:47.961011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:48.023456 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1380/2759] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:48.035700 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.035982 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.036252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.057446 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:48.058563 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:48.067044 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1380/2760] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:48.071561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.082060 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0df24-8381-4a9e-970a-a23551d5f52b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.087053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.087698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.087968 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.088231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.088881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:48.089178 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.090785 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.104489 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1381/2761] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:48.108994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.119343 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00544594-a187-4381-a9de-ed0cebc86340 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.125871 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.126564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.126927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.127366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.128110 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.130082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.174357 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1381/2762] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:48 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:48.180627 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.192059 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d6817-a0a9-4810-bd70-0bb94723c460 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.235513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:48.261513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.262135 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.262404 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.262668 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.263352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.265351 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.287983 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1382/2763] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:48 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:48.292205 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.302793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d350-25da-4ef5-a198-9b693ccc0236 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.307574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.308116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.308389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.308656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.309292 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.310913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.323710 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1382/2764] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:48.327945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.338286 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdeb52-9e0d-45c2-8762-b9334d07ac32 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.342840 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5035cda-2183-41dd-88d0-121e305910a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.343821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035cda-2183-41dd-88d0-121e305910a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.344114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035cda-2183-41dd-88d0-121e305910a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.344354 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5035cda-2183-41dd-88d0-121e305910a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.344979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035cda-2183-41dd-88d0-121e305910a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.346557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5035cda-2183-41dd-88d0-121e305910a9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.357420 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1383/2765] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:48.361821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.372420 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e777167-757f-4b95-9093-73d4538fd334 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.377680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.377932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.378171 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.378570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] PATH_INFO: `/v3/projects/5d8fe9c1759f4b52b83110351d2a31eb/users/fbbbcaddfb6a48efa43629988623cdb9/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.394444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8fe9c1759f4b52b83110351d2a31eb, user_id=fbbbcaddfb6a48efa43629988623cdb9, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.397247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.412599 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1383/2766] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:48 2022] PUT /identity/v3/projects/5d8fe9c1759f4b52b83110351d2a31eb/users/fbbbcaddfb6a48efa43629988623cdb9/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:48.424309 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.424741 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.425003 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.455282 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] MFA Rules not processed for user `fbbbcaddfb6a48efa43629988623cdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:48.456749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:48.525095 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1384/2767] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:48.537466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:48.539464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:48.580739 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:48.618333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f41af-2e6e-4994-a042-8e8bec20c046 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:48.625637 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eaca43-8188-42d8-a324-f08816bd18c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:48.626375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaca43-8188-42d8-a324-f08816bd18c9 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:48.626744 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaca43-8188-42d8-a324-f08816bd18c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:48.627107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eaca43-8188-42d8-a324-f08816bd18c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:48.636519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaca43-8188-42d8-a324-f08816bd18c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:48.638215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eaca43-8188-42d8-a324-f08816bd18c9 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:48.647652 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1384/2768] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:49.901312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:49.903050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:49.976478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff240f28-8f90-4664-a5cf-3f3d1380d3bb None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:49.980992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:49.981615 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:49.981881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:49.982142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] PATH_INFO: `/v3/users/98ed8708b2f145998d45f8d1ae45bd89` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:49.983260 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ed8708b2f145998d45f8d1ae45bd89)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:49.985277 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:50.044804 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ed8708b2f145998d45f8d1ae45bd89'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.049509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:50.056279 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ed8708b2f145998d45f8d1ae45bd89'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.073910 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ed8708b2f145998d45f8d1ae45bd89'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.074678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ed8708b2f145998d45f8d1ae45bd89'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.083838 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ed8708b2f145998d45f8d1ae45bd89'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.091046 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1385/2769] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:49 2022] DELETE /identity/v3/users/98ed8708b2f145998d45f8d1ae45bd89 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:50.095644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.096144 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.096476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.125579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:50.126732 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:50.190098 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1385/2770] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:50.200515 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:50.201934 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:50.282412 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b564015-2593-4816-9200-50e34a185d8b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:50.288941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd1b20-1bbf-4919-b976-d63d526f7675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:50.289587 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1b20-1bbf-4919-b976-d63d526f7675 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.289858 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1b20-1bbf-4919-b976-d63d526f7675 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.290143 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1b20-1bbf-4919-b976-d63d526f7675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.304716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd1b20-1bbf-4919-b976-d63d526f7675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:50.306428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd1b20-1bbf-4919-b976-d63d526f7675 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:50.328943 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1386/2771] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:50.696114 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:50.697938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:50.715051 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b4970-35a1-4944-8b0d-522421b002ea None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:50.720693 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:50.721333 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.721670 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.721948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] PATH_INFO: `/v3/projects/7601b75add294ad0af090dfbe639e5fb` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.723101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7601b75add294ad0af090dfbe639e5fb)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:50.725188 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:50.774362 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7601b75add294ad0af090dfbe639e5fb', 'request_id': 'req-1a76fd5e-80f4-488b-af83-a78b3d6ae342'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.783315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7601b75add294ad0af090dfbe639e5fb', 'request_id': 'req-1a76fd5e-80f4-488b-af83-a78b3d6ae342'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.791677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7601b75add294ad0af090dfbe639e5fb', 'request_id': 'req-1a76fd5e-80f4-488b-af83-a78b3d6ae342'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.802560 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] The token cache is being invalidate because project 7601b75add294ad0af090dfbe639e5fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:50.802845 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.803503 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-1a76fd5e-80f4-488b-af83-a78b3d6ae342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:50.805023 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1386/2772] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:50 2022] DELETE /identity/v3/projects/7601b75add294ad0af090dfbe639e5fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:50.809987 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.810334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.810689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.830407 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:50.831630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:50.893363 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1387/2773] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:50.900965 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.901278 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.901554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.922645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:50.923884 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:50.932764 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1387/2774] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:50.937274 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:50.947026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45fab-b9eb-4cf1-b0b6-c6f0e0a1f4f1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:50.951941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:50.952548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.952789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.953054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.953693 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:50.953980 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:50.955635 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:50.967329 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1388/2775] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:50.976241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:50.985828 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e646e0e-30d7-4042-bed7-1827ce178503 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:50.990640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:50.991268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:50.991535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:50.991798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:50.992423 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:50.994336 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:51.022655 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1388/2776] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:50 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:51.027002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:51.038065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bbe8e2-10e0-4717-9bb2-42ed21ac92b4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:51.092247 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:51.099571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:51.100189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:51.100588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:51.100851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:51.101487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:51.103488 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:51.125885 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1389/2777] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:51 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:51.130147 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:51.141019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4216a-2e02-4f19-99eb-9480a35cc58f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:51.146216 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:51.146752 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:51.147025 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:51.147456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:51.148129 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:51.149760 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:51.163239 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1389/2778] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:51.165095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:51.175955 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babba308-ede0-404d-ac24-8a77bc3acf4f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:51.181479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e861c305-0f6f-4661-a06a-a804c8b8b340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:51.182065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861c305-0f6f-4661-a06a-a804c8b8b340 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:51.182329 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861c305-0f6f-4661-a06a-a804c8b8b340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:51.182584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861c305-0f6f-4661-a06a-a804c8b8b340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:51.183227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861c305-0f6f-4661-a06a-a804c8b8b340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:51.184810 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e861c305-0f6f-4661-a06a-a804c8b8b340 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:51.195721 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1390/2779] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:51.199905 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:51.206643 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:51.210769 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92f2f5c-448b-4ba1-b096-27c8ce512450 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:51.217793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:51.218488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:51.218958 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:51.219268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] PATH_INFO: `/v3/projects/b755d83f0f5d4a20a7bab0be379afec3/users/6baedc377e104c789dc4a5b909bd07f4/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:51.234667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b755d83f0f5d4a20a7bab0be379afec3, user_id=6baedc377e104c789dc4a5b909bd07f4, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:51.237386 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:51.252985 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1390/2780] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:51 2022] PUT /identity/v3/projects/b755d83f0f5d4a20a7bab0be379afec3/users/6baedc377e104c789dc4a5b909bd07f4/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:51.263576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:51.263859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:51.264120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:51.294128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] MFA Rules not processed for user `6baedc377e104c789dc4a5b909bd07f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:51.295386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:51.371362 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1391/2781] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:51.383033 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:51.385083 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:51.445530 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b3a54-7d84-4876-814b-5c78a383cc0f admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:51.449402 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116b42e-1ce8-40e5-9d3e-c756d32fa7fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:51.449996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116b42e-1ce8-40e5-9d3e-c756d32fa7fd service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:51.450265 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116b42e-1ce8-40e5-9d3e-c756d32fa7fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:51.450528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116b42e-1ce8-40e5-9d3e-c756d32fa7fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:51.459759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116b42e-1ce8-40e5-9d3e-c756d32fa7fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:51.461166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116b42e-1ce8-40e5-9d3e-c756d32fa7fd service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:51.470269 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1391/2782] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:54.024655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:54.024655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.093431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed152b66-e26e-441c-b579-1285cce449c3 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:54.098527 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:54.099150 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.099463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.099727 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] PATH_INFO: `/v3/users/fbbbcaddfb6a48efa43629988623cdb9` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.100943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbbcaddfb6a48efa43629988623cdb9)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:54.102852 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:54.161365 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbbcaddfb6a48efa43629988623cdb9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.170794 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbbcaddfb6a48efa43629988623cdb9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.181851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:54.183964 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbbcaddfb6a48efa43629988623cdb9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.184745 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbbcaddfb6a48efa43629988623cdb9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.193400 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bf8a6d7d-a927-45b9-8098-c17036c2d343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbbcaddfb6a48efa43629988623cdb9'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.199246 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1392/2783] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:54 2022] DELETE /identity/v3/users/fbbbcaddfb6a48efa43629988623cdb9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:54.203995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.204502 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.204674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.224685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:54.225995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.289567 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1392/2784] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:54.298972 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.299450 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.299724 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.299920 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:54.301387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.324048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:54.341002 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.368011 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0aa21-5eaf-4ebc-987f-3aa15c63d571 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:54.372543 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:54.373549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.374200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.374893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.392528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:54.394290 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:54.405145 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1393/2785] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:54.407524 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1393/2786] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:54.426932 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:54.428479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.463341 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:54.509942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c4bfc-648a-4603-81b2-5bed131b36de service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:54.516527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3a8d-bd4d-42d5-940b-482aca0188a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:54.517157 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3a8d-bd4d-42d5-940b-482aca0188a3 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.517468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3a8d-bd4d-42d5-940b-482aca0188a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.517730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3a8d-bd4d-42d5-940b-482aca0188a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.527742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d3a8d-bd4d-42d5-940b-482aca0188a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:54.529720 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d3a8d-bd4d-42d5-940b-482aca0188a3 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:54.541196 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1394/2787] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:02:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 01:02:54.750369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:54.752584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.786254 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe5f7b-1db0-44c1-a65f-d52640037fea None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:54.791720 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:54.792295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.792601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.792814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] PATH_INFO: `/v3/projects/5d8fe9c1759f4b52b83110351d2a31eb` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.793991 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8fe9c1759f4b52b83110351d2a31eb)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:54.795913 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:54.849473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8fe9c1759f4b52b83110351d2a31eb', 'request_id': 'req-9b5aacb9-428f-45f9-929d-7b6daa83c085'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.857460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8fe9c1759f4b52b83110351d2a31eb', 'request_id': 'req-9b5aacb9-428f-45f9-929d-7b6daa83c085'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.866645 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8fe9c1759f4b52b83110351d2a31eb', 'request_id': 'req-9b5aacb9-428f-45f9-929d-7b6daa83c085'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.878780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] The token cache is being invalidate because project 5d8fe9c1759f4b52b83110351d2a31eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:54.879171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.879904 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-9b5aacb9-428f-45f9-929d-7b6daa83c085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:54.881410 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1394/2788] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:54 2022] DELETE /identity/v3/projects/5d8fe9c1759f4b52b83110351d2a31eb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:54.886291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.886569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.886838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:54.907667 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:54.909017 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:54.971521 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1395/2789] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:54.980601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:54.980943 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:54.981295 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.003667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:55.004859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:55.012726 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1395/2790] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:55.017042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.026895 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb35c4c-5a45-43fe-b203-8fed07de7698 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.031697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.032457 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.032844 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.033254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.034092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:02:55.034449 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.036214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.053589 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1396/2791] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:02:55.054721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.064048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989c240-6d7e-47c9-b15b-f867f26809c2 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.069502 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e5cb1-34ba-4b06-a9fe-69f1e0e20a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.070129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5cb1-34ba-4b06-a9fe-69f1e0e20a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.070424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5cb1-34ba-4b06-a9fe-69f1e0e20a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.070719 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e5cb1-34ba-4b06-a9fe-69f1e0e20a3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.071482 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e5cb1-34ba-4b06-a9fe-69f1e0e20a3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.073557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e5cb1-34ba-4b06-a9fe-69f1e0e20a3a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.105876 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1396/2792] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:55.107031 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.118439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983943b-9c3b-4825-bcde-b522866fa02b admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.131929 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.132202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.132464 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.152063 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:55.153320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:55.183639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.184261 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.184540 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.184809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.185603 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.187628 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.197311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:55.210861 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1397/2793] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:55.215062 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.217967 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1397/2794] 10.222.0.23 () {58 vars in 967 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:55.226018 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b7598-b11b-404d-935c-89ddc2a57c22 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.230563 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.231014 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.231655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.231930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.232058 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:55.232230 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.232838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.234545 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.248325 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1398/2795] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:55.249318 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.258875 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d5702-28cd-4cb4-ad25-15d39a9781e2 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.263542 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.264123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.264403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.264675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.265320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.266983 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.277108 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1399/2796] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:02:55.280886 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.290677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29936b-dd61-4f6b-9e5a-9b558f66d6b5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.295351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee5fa2-ae72-4a6f-b7b1-4ff2f6747c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.295974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee5fa2-ae72-4a6f-b7b1-4ff2f6747c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.296247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee5fa2-ae72-4a6f-b7b1-4ff2f6747c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.296513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee5fa2-ae72-4a6f-b7b1-4ff2f6747c6e admin admin] PATH_INFO: `/v3/projects/047a01569d2648e68118143c40915281/users/8eea149e488d4fb093a59ae3486df166/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.304055 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5668f7-ba58-49e6-bf20-ffb7d5db20f1 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.304968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:55.309881 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.310555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.310846 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.311118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.312897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee5fa2-ae72-4a6f-b7b1-4ff2f6747c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=047a01569d2648e68118143c40915281, user_id=8eea149e488d4fb093a59ae3486df166, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.315624 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee5fa2-ae72-4a6f-b7b1-4ff2f6747c6e admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.319628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.321471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.328763 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1400/2797] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:55 2022] PUT /identity/v3/projects/047a01569d2648e68118143c40915281/users/8eea149e488d4fb093a59ae3486df166/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:55.329543 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1398/2798] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 01:02:55.339197 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.339610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.339819 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.370549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] MFA Rules not processed for user `8eea149e488d4fb093a59ae3486df166`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:55.371692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:55.443263 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1401/2799] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:55.447935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.458423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19b071-8e86-4758-ad55-58de153e2b2f service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.520783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfaf25-503f-47d6-8369-813c582d62b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.521448 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfaf25-503f-47d6-8369-813c582d62b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.521660 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfaf25-503f-47d6-8369-813c582d62b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.521920 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfaf25-503f-47d6-8369-813c582d62b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.522558 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfaf25-503f-47d6-8369-813c582d62b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.524655 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcfaf25-503f-47d6-8369-813c582d62b2 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.553882 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1399/2800] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:02:55.558109 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.567549 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.567835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.568109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.569606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac5bba-59e5-4e46-851b-d286dbd672ff None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.587856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:55.589065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:55.630072 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.630681 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.630957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.631519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.631881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.633780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.646175 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1400/2801] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:55.658259 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.658974 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1402/2802] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:02:55.663254 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.668004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:55.674134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee8647-9b20-4771-8c90-83972a2e3fd9 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.679053 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.679649 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.679927 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.680192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.680833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.682413 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.693327 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1403/2803] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:55.698099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.710117 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9ec56b-86a1-46c1-827c-bb94d53c9c09 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.714969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.715650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.715923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.716189 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] PATH_INFO: `/v3/projects/6e17feea127c4a49a8e7a85617b76739/users/e1bb850b30fb470f92c2a7c7bd670209/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.729084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc9549-b7cd-4471-a45e-32a7d3eb371b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.731444 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e17feea127c4a49a8e7a85617b76739, user_id=e1bb850b30fb470f92c2a7c7bd670209, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.734154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.734275 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.734860 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.735134 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.735437 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.744228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.745976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.747803 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1404/2804] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:55 2022] PUT /identity/v3/projects/6e17feea127c4a49a8e7a85617b76739/users/e1bb850b30fb470f92c2a7c7bd670209/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:02:55.752479 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.754875 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1401/2805] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 11 01:02:55.796118 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:55.832569 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8ce50-752a-4644-8d60-c9b73027f3bf admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.838353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd959ef-fa50-45e5-b5ef-275811cf5e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.838995 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd959ef-fa50-45e5-b5ef-275811cf5e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.839375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd959ef-fa50-45e5-b5ef-275811cf5e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.839644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd959ef-fa50-45e5-b5ef-275811cf5e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.840291 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd959ef-fa50-45e5-b5ef-275811cf5e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.841870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd959ef-fa50-45e5-b5ef-275811cf5e76 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.853061 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1405/2806] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:02:55.864357 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:55.875812 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d38d15-c594-47d1-a269-98d2302ae01f service swift] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:55.884822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:55.885572 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.885912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.886245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] PATH_INFO: `/v3/projects/6e17feea127c4a49a8e7a85617b76739/users/e1bb850b30fb470f92c2a7c7bd670209/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.888790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e17feea127c4a49a8e7a85617b76739, user_id=e1bb850b30fb470f92c2a7c7bd670209, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:55.891171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:55.905496 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1402/2807] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:02:55 2022] PUT /identity/v3/projects/6e17feea127c4a49a8e7a85617b76739/users/e1bb850b30fb470f92c2a7c7bd670209/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:55.916850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a709a-02c7-4f5c-9c49-1fadc7bf83c3 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:55.917226 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a709a-02c7-4f5c-9c49-1fadc7bf83c3 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:55.917595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a709a-02c7-4f5c-9c49-1fadc7bf83c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:55.947837 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1f2a709a-02c7-4f5c-9c49-1fadc7bf83c3 None None] MFA Rules not processed for user `e1bb850b30fb470f92c2a7c7bd670209`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:55.949293 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1f2a709a-02c7-4f5c-9c49-1fadc7bf83c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:56.034836 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1406/2808] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:56.096588 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:56.098484 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:56.131092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:56.131427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:56.131712 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:56.151685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:56.153671 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:56.184640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cfa69-5326-420d-98d2-b984beb9ee3c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:56.189722 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:56.190583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:56.190863 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:56.191305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:56.216209 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1407/2809] 10.222.0.23 () {58 vars in 967 bytes} [Sun Sep 11 01:02:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:02:56.217201 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:56.220673 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:56.229780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:56.235375 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1403/2810] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:56.299360 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=100 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:56.317912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9cadc6-5e59-419e-a0c3-f7951768db62 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:56.329759 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:56.330493 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:56.330850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:56.331112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:56.334435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:56.341535 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:56.343162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:56.345814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79bf64-94cd-4af0-a6c1-0a0b724c413b service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:56.350257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e01534-725f-4de1-b5b4-bde08c49c84d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:56.351136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01534-725f-4de1-b5b4-bde08c49c84d service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:56.351705 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01534-725f-4de1-b5b4-bde08c49c84d service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:56.352443 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1408/2811] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:02:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 01:02:56.376094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e01534-725f-4de1-b5b4-bde08c49c84d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:56.472169 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e01534-725f-4de1-b5b4-bde08c49c84d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:56.474954 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e01534-725f-4de1-b5b4-bde08c49c84d service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:56.490310 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1404/2812] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:02:59.086236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:59.087831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:59.160464 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876e847-3488-4eb9-b6cc-d24d650c6d25 service swift] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:59.167660 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:59.168299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:59.168570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:59.168835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] PATH_INFO: `/v3/users/74cd83f1bed24ac48ea36b91613f021c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:59.169969 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74cd83f1bed24ac48ea36b91613f021c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:59.171923 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:59.225537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74cd83f1bed24ac48ea36b91613f021c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.240977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74cd83f1bed24ac48ea36b91613f021c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.250612 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74cd83f1bed24ac48ea36b91613f021c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.251451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74cd83f1bed24ac48ea36b91613f021c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.264433 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74cd83f1bed24ac48ea36b91613f021c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.271061 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1409/2813] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:02:59 2022] DELETE /identity/v3/users/74cd83f1bed24ac48ea36b91613f021c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:02:59.276263 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:59.276766 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:59.276865 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:59.303537 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:02:59.305749 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:59.346509 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:02:59.412766 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1405/2814] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:02:59.423283 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:59.424859 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:59.532833 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6e09-381f-4af9-a17b-d11a81f8d281 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:59.539798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:59.540764 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:59.541334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:59.541785 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:59.553771 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:59.555883 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:59.586660 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1410/2815] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:02:59.820787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:02:59.822303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:02:59.838311 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77d351-6908-46d6-8719-756ed71cff5f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:02:59.845390 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:02:59.846343 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:02:59.846733 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:02:59.847081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] PATH_INFO: `/v3/projects/eecf50e62b474fa1ae29cf0197bdefe7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:02:59.848995 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eecf50e62b474fa1ae29cf0197bdefe7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:02:59.851074 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:02:59.901562 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eecf50e62b474fa1ae29cf0197bdefe7', 'request_id': 'req-b0d98519-f1b5-41f3-9623-14763afee433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.911842 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eecf50e62b474fa1ae29cf0197bdefe7', 'request_id': 'req-b0d98519-f1b5-41f3-9623-14763afee433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.918494 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eecf50e62b474fa1ae29cf0197bdefe7', 'request_id': 'req-b0d98519-f1b5-41f3-9623-14763afee433'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.932603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] The token cache is being invalidate because project eecf50e62b474fa1ae29cf0197bdefe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:02:59.932887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.933536 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-b0d98519-f1b5-41f3-9623-14763afee433 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:02:59.935259 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1406/2816] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:02:59 2022] DELETE /identity/v3/projects/eecf50e62b474fa1ae29cf0197bdefe7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:02.239630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:02.241305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:02.255528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97175f31-8cad-45b3-9015-04949712ef08 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:02.261489 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:02.317550 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:02.318182 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:02.318459 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:02.318728 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] PATH_INFO: `/v3/users/6baedc377e104c789dc4a5b909bd07f4` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:02.319893 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6baedc377e104c789dc4a5b909bd07f4)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:02.321780 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:02.373452 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6baedc377e104c789dc4a5b909bd07f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:02.382002 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6baedc377e104c789dc4a5b909bd07f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:02.389151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6baedc377e104c789dc4a5b909bd07f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:02.389804 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6baedc377e104c789dc4a5b909bd07f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:02.401554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6baedc377e104c789dc4a5b909bd07f4'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:02.407367 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1411/2817] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:02 2022] DELETE /identity/v3/users/6baedc377e104c789dc4a5b909bd07f4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:02.412511 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:02.413001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:02.413001 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:02.433442 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:02.434642 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:02.500410 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1407/2818] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:02.511315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:02.512779 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:02.585332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cf379-0d1c-4781-9741-91ec216fa934 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:02.589526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:02.590111 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:02.590441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:02.590724 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:02.600123 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:02.601788 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:02.612533 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1412/2819] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:02.914480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:02.916011 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:02.928626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c00699-02fb-4266-a0aa-3e76b0793676 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:02.934640 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:02.935204 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:02.935543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:02.935811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] PATH_INFO: `/v3/projects/b755d83f0f5d4a20a7bab0be379afec3` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:02.937084 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b755d83f0f5d4a20a7bab0be379afec3)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:02.939202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:02.992296 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:02.995602 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b755d83f0f5d4a20a7bab0be379afec3', 'request_id': 'req-2be87e25-7e32-4a96-bf33-17360a058721'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.004438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b755d83f0f5d4a20a7bab0be379afec3', 'request_id': 'req-2be87e25-7e32-4a96-bf33-17360a058721'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.010009 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b755d83f0f5d4a20a7bab0be379afec3', 'request_id': 'req-2be87e25-7e32-4a96-bf33-17360a058721'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.021807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] The token cache is being invalidate because project b755d83f0f5d4a20a7bab0be379afec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:03.022112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.022786 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2be87e25-7e32-4a96-bf33-17360a058721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.024460 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1408/2820] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:02 2022] DELETE /identity/v3/projects/b755d83f0f5d4a20a7bab0be379afec3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:03.721586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:03.722996 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:03.739827 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b01f1a-930c-4140-b3a1-1c280addcc2c service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:03.782776 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:03.831329 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:03.831987 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:03.832257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:03.832521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] PATH_INFO: `/v3/users/8eea149e488d4fb093a59ae3486df166` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:03.833656 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eea149e488d4fb093a59ae3486df166)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:03.835597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:03.884742 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eea149e488d4fb093a59ae3486df166'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.894366 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eea149e488d4fb093a59ae3486df166'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.902322 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eea149e488d4fb093a59ae3486df166'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.903277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eea149e488d4fb093a59ae3486df166'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.919419 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eea149e488d4fb093a59ae3486df166'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:03.927069 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1413/2821] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:03 2022] DELETE /identity/v3/users/8eea149e488d4fb093a59ae3486df166 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:03.931761 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:03.932300 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:03.932487 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:03.956453 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:03.958053 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:04.021135 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1409/2822] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:04.031155 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:04.032680 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:04.100309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b9825-87f4-4340-997d-d54cd27d3a50 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:04.104100 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:04.104716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:04.104988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:04.105253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:04.115611 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:04.117148 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:04.129018 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1414/2823] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:04.440436 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:04.441952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:04.454601 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dad5e4-2595-4a37-8e6f-bba3727def55 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:04.459581 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:04.460212 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:04.460480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:04.460787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] PATH_INFO: `/v3/projects/047a01569d2648e68118143c40915281` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:04.461885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=047a01569d2648e68118143c40915281)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:04.463805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:04.510102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047a01569d2648e68118143c40915281', 'request_id': 'req-e346be85-ec6f-495d-aeff-6d43f9c74a1a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.519851 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047a01569d2648e68118143c40915281', 'request_id': 'req-e346be85-ec6f-495d-aeff-6d43f9c74a1a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.525416 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047a01569d2648e68118143c40915281', 'request_id': 'req-e346be85-ec6f-495d-aeff-6d43f9c74a1a'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.536612 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] The token cache is being invalidate because project 047a01569d2648e68118143c40915281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:04.536894 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.537559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-e346be85-ec6f-495d-aeff-6d43f9c74a1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.539084 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1410/2824] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:04 2022] DELETE /identity/v3/projects/047a01569d2648e68118143c40915281 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:04.543210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:04.553239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6db935-453b-4b9f-9146-665cc0c72326 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:04.602050 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:04.614401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:04.615219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:04.615403 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:04.615617 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] PATH_INFO: `/v3/users/e1bb850b30fb470f92c2a7c7bd670209` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:04.616960 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1bb850b30fb470f92c2a7c7bd670209)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:04.618797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:04.675811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1bb850b30fb470f92c2a7c7bd670209'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.685279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1bb850b30fb470f92c2a7c7bd670209'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.692821 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1bb850b30fb470f92c2a7c7bd670209'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.693533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1bb850b30fb470f92c2a7c7bd670209'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.707026 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1bb850b30fb470f92c2a7c7bd670209'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:04.713303 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1415/2825] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:04 2022] DELETE /identity/v3/users/e1bb850b30fb470f92c2a7c7bd670209 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:04.921240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:04.921521 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:04.921935 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:04.941583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] MFA Rules not processed for user `8f67555ada824965b62e7026cfbab0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:04.942854 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:04.999645 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.001781 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1411/2826] 10.222.0.23 () {58 vars in 967 bytes} [Sun Sep 11 01:03:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:05.002380 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:05.028576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.030132 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:05.035036 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:05.087437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9c752-ac64-4607-8adf-d097a2e2d507 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.092299 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.092931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.093227 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.093521 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] PATH_INFO: `/v3/projects/6e17feea127c4a49a8e7a85617b76739` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.094765 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e17feea127c4a49a8e7a85617b76739)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.096981 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.104056 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244d5bd-152e-46e7-abdd-1ff5e1ccc86c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.109031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d1b86-90bf-4a80-9c55-ba4e8881a266 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '512080b44c7542439333bd8b5f941496', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.109629 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1b86-90bf-4a80-9c55-ba4e8881a266 service swift] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.109903 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1b86-90bf-4a80-9c55-ba4e8881a266 service swift] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.110165 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d1b86-90bf-4a80-9c55-ba4e8881a266 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.118164 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d1b86-90bf-4a80-9c55-ba4e8881a266 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.119667 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d1b86-90bf-4a80-9c55-ba4e8881a266 service swift] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.128520 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1412/2827] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Sep 11 01:03:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 11 01:03:05.147367 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e17feea127c4a49a8e7a85617b76739', 'request_id': 'req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:05.157564 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e17feea127c4a49a8e7a85617b76739', 'request_id': 'req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:05.164805 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e17feea127c4a49a8e7a85617b76739', 'request_id': 'req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:05.175909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] The token cache is being invalidate because project 6e17feea127c4a49a8e7a85617b76739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:05.176345 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:05.177121 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-a2ba6ee4-591d-4cec-9df9-8a12a49a5449 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:05.178680 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1416/2828] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:04 2022] DELETE /identity/v3/projects/6e17feea127c4a49a8e7a85617b76739 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:05.183750 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.184029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.184291 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.210497 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:05.211760 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:05.275211 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1413/2829] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:03:05.284662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.285079 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.285358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.294039 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:05.308625 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:05.309818 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:05.318165 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1417/2830] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:05.322427 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.332369 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4472f8-9577-40f5-834a-f97191e33577 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.337081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.337688 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.337963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.338232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.338867 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:03:05.339163 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.340769 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.352828 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1414/2831] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 11 01:03:05.357793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.367686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466aa7c-bdca-426b-a363-eac461325ed1 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.372334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.372943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.373219 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.373482 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.374268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.376458 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.408267 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1418/2832] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:05.409721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.420188 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8231080f-f85a-4a1c-bfa7-cdbec034d55e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.477755 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.478582 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.478950 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.479342 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.480079 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.482106 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.504805 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1415/2833] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:05.508841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.519395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be196bc-ac07-42e0-aed6-9dc28be2adae admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.524003 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.524675 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.525008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.525344 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.526054 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.527686 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.538794 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1419/2834] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:05.542716 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.553862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d05223-a079-42c0-9fa5-7bdd44c7b3c1 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.559378 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc8721-e70b-4200-8142-6c5f4b25f2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.559985 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc8721-e70b-4200-8142-6c5f4b25f2ff admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.560227 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc8721-e70b-4200-8142-6c5f4b25f2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.560490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc8721-e70b-4200-8142-6c5f4b25f2ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.561183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc8721-e70b-4200-8142-6c5f4b25f2ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.562807 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc8721-e70b-4200-8142-6c5f4b25f2ff admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.565156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:05.579358 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1416/2835] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:05.580038 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.591897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c23c-95b3-4567-8600-966820545cd3 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.597488 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.598143 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.598438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.598701 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] PATH_INFO: `/v3/projects/503fca47698544c485ab9efcaf7e4a39/users/d0e6ac203f794a3f8b784d9bab23bf70/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.613935 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=503fca47698544c485ab9efcaf7e4a39, user_id=d0e6ac203f794a3f8b784d9bab23bf70, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.616635 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.630902 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1420/2836] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:05 2022] PUT /identity/v3/projects/503fca47698544c485ab9efcaf7e4a39/users/d0e6ac203f794a3f8b784d9bab23bf70/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:05.641393 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.641686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.641948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.671868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] MFA Rules not processed for user `d0e6ac203f794a3f8b784d9bab23bf70`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:05.673107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:05.744636 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1417/2837] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:05.749250 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.812915 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78190d-d59f-4d86-858e-c5df4d882ac1 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.817501 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.818139 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.818415 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.818674 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.819337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.821240 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.849403 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1421/2838] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:05.853518 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.864478 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910fc7dd-bc7a-4fa1-b18c-2740468c2598 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.921532 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.922312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.922584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.922859 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.923550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.925528 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.948034 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1418/2839] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:05 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:05.952080 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.962418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904882a-644b-458c-87c9-acab4f378f80 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:05.968229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:05.968891 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:05.969305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:05.969574 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:05.970359 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:05.971997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:05.982840 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1422/2840] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:05.987108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:05.997189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d3d9c5-f7f9-4a24-91e2-2ad8ec88a96a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:06.002161 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:06.002897 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.003271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.003545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] PATH_INFO: `/v3/projects/14f9807832d04174863f8675199bd3b0/users/4e77833ea09949d1a75e1a0905aad32f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:06.020224 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f9807832d04174863f8675199bd3b0, user_id=4e77833ea09949d1a75e1a0905aad32f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:06.022848 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:06.038241 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1419/2841] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:05 2022] PUT /identity/v3/projects/14f9807832d04174863f8675199bd3b0/users/4e77833ea09949d1a75e1a0905aad32f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:06.043102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:06.053484 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412352c-5314-43e5-9c51-623a48e53b20 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:06.114463 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436f30a-2da0-4c4e-b3ec-d7925a1f6c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:06.115153 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436f30a-2da0-4c4e-b3ec-d7925a1f6c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.115523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436f30a-2da0-4c4e-b3ec-d7925a1f6c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.115864 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436f30a-2da0-4c4e-b3ec-d7925a1f6c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:06.116579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436f30a-2da0-4c4e-b3ec-d7925a1f6c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:06.118185 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436f30a-2da0-4c4e-b3ec-d7925a1f6c00 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:06.131856 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1423/2842] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:06.133380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:06.147005 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa6402-05e0-4b10-9975-20d8e3775eaa admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:06.152334 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:06.153116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.153396 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.153685 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] PATH_INFO: `/v3/projects/14f9807832d04174863f8675199bd3b0/users/4e77833ea09949d1a75e1a0905aad32f/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:06.155619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f9807832d04174863f8675199bd3b0, user_id=4e77833ea09949d1a75e1a0905aad32f, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:06.158477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:06.169686 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1420/2843] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:06 2022] PUT /identity/v3/projects/14f9807832d04174863f8675199bd3b0/users/4e77833ea09949d1a75e1a0905aad32f/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:06.180921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.181133 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.181406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:06.201579 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:06.211727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] MFA Rules not processed for user `4e77833ea09949d1a75e1a0905aad32f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:06.213400 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:06.284894 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1424/2844] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:06.296546 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:06.298371 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:06.347326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:06.361849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13efe9d-f5dc-4734-927a-abfca1c04e5b admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:06.366997 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d046c0a-0e9e-4611-8aaa-59c3e616c21f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:06.367707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d046c0a-0e9e-4611-8aaa-59c3e616c21f service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.368046 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d046c0a-0e9e-4611-8aaa-59c3e616c21f service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.368210 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d046c0a-0e9e-4611-8aaa-59c3e616c21f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:06.380127 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d046c0a-0e9e-4611-8aaa-59c3e616c21f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:06.381616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d046c0a-0e9e-4611-8aaa-59c3e616c21f service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:06.394573 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1421/2845] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:06.828838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:06.891358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b4bb6-d58e-4567-8565-e5e6d85f9204 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:06.896567 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:06.897207 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.897476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.897740 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] PATH_INFO: `/v3/users/d0e6ac203f794a3f8b784d9bab23bf70` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:06.898947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e6ac203f794a3f8b784d9bab23bf70)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:06.900912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:06.951518 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e6ac203f794a3f8b784d9bab23bf70'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:06.959578 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e6ac203f794a3f8b784d9bab23bf70'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:06.966526 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e6ac203f794a3f8b784d9bab23bf70'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:06.967149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e6ac203f794a3f8b784d9bab23bf70'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:06.979042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e6ac203f794a3f8b784d9bab23bf70'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:06.984950 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1425/2846] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:06 2022] DELETE /identity/v3/users/d0e6ac203f794a3f8b784d9bab23bf70 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:06.989570 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:06.990403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:06.990776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:07.009384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:07.010726 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:07.069763 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1422/2847] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:07.079781 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:07.081389 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:07.096442 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:07.163274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4682a-1f95-4ab8-ae2c-dbaf64b6506a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:07.167773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:07.168474 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:07.168816 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:07.169151 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:07.180862 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:07.182931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:07.193635 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1426/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:07.506790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:07.508637 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:07.532386 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160eab16-6b9f-4334-8b5c-ef8c55c965f8 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:07.548223 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:07.548891 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:07.549198 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:07.549465 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] PATH_INFO: `/v3/projects/503fca47698544c485ab9efcaf7e4a39` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:07.550622 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=503fca47698544c485ab9efcaf7e4a39)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:07.552566 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:07.605783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '503fca47698544c485ab9efcaf7e4a39', 'request_id': 'req-bcbcadb7-692b-424e-ad39-9e6f136fa492'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.615240 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '503fca47698544c485ab9efcaf7e4a39', 'request_id': 'req-bcbcadb7-692b-424e-ad39-9e6f136fa492'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.620993 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '503fca47698544c485ab9efcaf7e4a39', 'request_id': 'req-bcbcadb7-692b-424e-ad39-9e6f136fa492'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.636674 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] The token cache is being invalidate because project 503fca47698544c485ab9efcaf7e4a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:07.636963 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.637630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.640126 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1423/2849] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:07 2022] DELETE /identity/v3/projects/503fca47698544c485ab9efcaf7e4a39 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:07.644395 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:07.658925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ab6c1-263c-414e-8e48-0e8bcc7a8947 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:07.723692 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:07.724248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:07.724528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:07.724851 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] PATH_INFO: `/v3/users/4e77833ea09949d1a75e1a0905aad32f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:07.726113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e77833ea09949d1a75e1a0905aad32f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:07.728103 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:07.778078 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e77833ea09949d1a75e1a0905aad32f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.787304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e77833ea09949d1a75e1a0905aad32f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.794536 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e77833ea09949d1a75e1a0905aad32f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.795167 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e77833ea09949d1a75e1a0905aad32f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.808868 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-65a8db4d-40e1-4ae2-b57b-cb2fe0c3f1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e77833ea09949d1a75e1a0905aad32f'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:07.815007 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1427/2850] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:07 2022] DELETE /identity/v3/users/4e77833ea09949d1a75e1a0905aad32f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:08.104225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.105675 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:08.127283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcadb7-692b-424e-ad39-9e6f136fa492 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.203380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.204167 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.204506 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.204836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] PATH_INFO: `/v3/projects/14f9807832d04174863f8675199bd3b0` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.206021 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f9807832d04174863f8675199bd3b0)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.208022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.216388 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:08.266298 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f9807832d04174863f8675199bd3b0', 'request_id': 'req-c7f7eb9e-4269-444f-8bca-771360e84e43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:08.276140 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f9807832d04174863f8675199bd3b0', 'request_id': 'req-c7f7eb9e-4269-444f-8bca-771360e84e43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:08.283070 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f9807832d04174863f8675199bd3b0', 'request_id': 'req-c7f7eb9e-4269-444f-8bca-771360e84e43'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:08.296787 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] The token cache is being invalidate because project 14f9807832d04174863f8675199bd3b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:08.297200 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:08.297754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-c7f7eb9e-4269-444f-8bca-771360e84e43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:08.299570 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1424/2851] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:08 2022] DELETE /identity/v3/projects/14f9807832d04174863f8675199bd3b0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:08.304561 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.305534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.305874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.327073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:08.328549 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:08.362947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:08.407874 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1428/2852] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:08.409435 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.409781 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.410120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.442294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:08.443626 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:08.455248 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1425/2853] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 11 01:03:08.459582 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.474952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f084f0-269f-4c5c-a855-760a2eb07e1d None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.479921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.480472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.480749 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.481282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.481952 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:03:08.482255 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.484089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.498831 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1429/2854] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:03:08.502927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.516100 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046198e-9345-410f-b23e-fcb56c4d360f None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.521289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.521930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.522203 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.522473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.523120 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.525141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.568932 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1426/2855] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:08 2022] POST /identity/v3/projects => generated 355 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:08.573357 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.585276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8ee3f-6062-4635-886d-0a11fd88336c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.739897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.744919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.744919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.744919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.744919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.744919 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.791680 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1430/2856] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:08 2022] POST /identity/v3/users => generated 278 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:08.795423 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.824489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943484-57b5-4445-95ba-188ee56213a7 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.829552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.830252 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.830592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.830927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.831687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.838648 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.852472 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1427/2857] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:08.859437 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.870973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613e9af-806f-4e94-9b4c-53adbf4b7815 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.875989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e093436-278b-4ed0-9237-c6844d924b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.876636 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e093436-278b-4ed0-9237-c6844d924b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.876989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e093436-278b-4ed0-9237-c6844d924b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.877320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e093436-278b-4ed0-9237-c6844d924b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.877976 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e093436-278b-4ed0-9237-c6844d924b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.879676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e093436-278b-4ed0-9237-c6844d924b84 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.891331 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1431/2858] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:08.895471 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:08.905960 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f27a9-a060-42c8-bd07-eec6d06b60d3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:08.911006 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:08.912561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.912832 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.913095 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] PATH_INFO: `/v3/projects/2dfc3d000db842ea8dafb4fb69a4166c/users/c9cff541ea324815b62b8710e7ce9dc6/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.928270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfc3d000db842ea8dafb4fb69a4166c, user_id=c9cff541ea324815b62b8710e7ce9dc6, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:08.931128 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:08.945152 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1428/2859] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:08 2022] PUT /identity/v3/projects/2dfc3d000db842ea8dafb4fb69a4166c/users/c9cff541ea324815b62b8710e7ce9dc6/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:08.955589 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:08.955869 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:08.956137 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:08.983577 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] MFA Rules not processed for user `c9cff541ea324815b62b8710e7ce9dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:08.984748 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:09.053567 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1432/2860] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:09.058272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.120687 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6897ba-207d-411e-aaaa-0c61333f5b3f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.125461 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.126067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.126346 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.126603 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.127265 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.129194 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.166205 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1429/2861] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:09 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:09.170203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.181228 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef662a8-fba0-4e96-b347-e22b8868f5ad None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.210204 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:09.240173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.240702 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.240973 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.241236 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.241882 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.243942 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.269960 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1433/2862] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:09 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 11 01:03:09.271525 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.282199 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8eb42f-6100-4bee-b67c-4d6c2aaded40 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.287273 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.287877 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.288154 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.288426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.289067 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.290659 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.301689 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1430/2863] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:09.305809 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.317158 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5eabe6-d3f2-4698-af6e-9558cd1f6757 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.322909 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.323607 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.323878 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.324141 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] PATH_INFO: `/v3/projects/8edc5007b81a49658f8b1a16c2fd8ab8/users/1f3c441381c845f9bd1e6a1587d41e26/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.339532 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edc5007b81a49658f8b1a16c2fd8ab8, user_id=1f3c441381c845f9bd1e6a1587d41e26, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.342169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.356738 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1434/2864] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:09 2022] PUT /identity/v3/projects/8edc5007b81a49658f8b1a16c2fd8ab8/users/1f3c441381c845f9bd1e6a1587d41e26/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:09.360779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.371292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af1c61-862b-496f-a81d-79faf20300ca admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.375513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:09.432454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98aca8-4108-4419-9f14-3e097671212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.433064 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98aca8-4108-4419-9f14-3e097671212d admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.433332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98aca8-4108-4419-9f14-3e097671212d admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.433593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98aca8-4108-4419-9f14-3e097671212d admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.434264 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98aca8-4108-4419-9f14-3e097671212d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.435912 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db98aca8-4108-4419-9f14-3e097671212d admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.445907 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1431/2865] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:09.450443 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.460639 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5760ef-37b2-4dd8-b9c3-7d5b23d0d0bc admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.465247 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.465897 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.466163 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.466438 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] PATH_INFO: `/v3/projects/8edc5007b81a49658f8b1a16c2fd8ab8/users/1f3c441381c845f9bd1e6a1587d41e26/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.468586 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edc5007b81a49658f8b1a16c2fd8ab8, user_id=1f3c441381c845f9bd1e6a1587d41e26, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.471347 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.482974 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1435/2866] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:09 2022] PUT /identity/v3/projects/8edc5007b81a49658f8b1a16c2fd8ab8/users/1f3c441381c845f9bd1e6a1587d41e26/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:09.494384 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.494598 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.494959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.524048 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] MFA Rules not processed for user `1f3c441381c845f9bd1e6a1587d41e26`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:09.525317 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:09.597320 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1432/2867] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:09.609221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.611134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:09.669391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e35d0-92d9-4334-9d0d-0d5b0c3f7e44 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.673135 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.673743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.674012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.674304 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.684258 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.685741 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.698314 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1436/2868] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:09.778213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:09.791004 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a01e12-d1dd-402b-8920-6f24fee7be5b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:09.795553 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:09.796183 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:09.796459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:09.796734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:09.862981 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:09.865136 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:09.876596 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1433/2869] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:12.096523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:12.098074 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:12.176441 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2c1cc0-a708-4929-ac70-7a971af4a6b4 service cinder] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:12.181375 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:12.182008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:12.182280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:12.182546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] PATH_INFO: `/v3/users/b0ba6f92b3d34bbeb1f7d52de32cd29c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:12.183673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ba6f92b3d34bbeb1f7d52de32cd29c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:12.185587 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:12.215200 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:12.240627 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ba6f92b3d34bbeb1f7d52de32cd29c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.249257 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ba6f92b3d34bbeb1f7d52de32cd29c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.263790 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ba6f92b3d34bbeb1f7d52de32cd29c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.264520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ba6f92b3d34bbeb1f7d52de32cd29c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.277989 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ba6f92b3d34bbeb1f7d52de32cd29c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.284901 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1437/2870] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:12 2022] DELETE /identity/v3/users/b0ba6f92b3d34bbeb1f7d52de32cd29c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:12.461054 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:12.483065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe813a-ba00-4915-9eb5-dc270318d658 service cinder] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:12.515130 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=100 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:12.552990 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:12.560425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:12.560425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:12.560425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] PATH_INFO: `/v3/projects/69a25da8ed8849bba71977e48ec7bd8d` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:12.560425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a25da8ed8849bba71977e48ec7bd8d)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:12.560425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:12.615789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a25da8ed8849bba71977e48ec7bd8d', 'request_id': 'req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.624977 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a25da8ed8849bba71977e48ec7bd8d', 'request_id': 'req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.634202 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a25da8ed8849bba71977e48ec7bd8d', 'request_id': 'req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.647678 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] The token cache is being invalidate because project 69a25da8ed8849bba71977e48ec7bd8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:12.647961 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.648608 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.650117 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1434/2871] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:12 2022] DELETE /identity/v3/projects/69a25da8ed8849bba71977e48ec7bd8d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:12.654595 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:12.656427 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:12.671004 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ddc98-6d85-4280-82cc-cf81fdc8ae11 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:12.732058 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:12.732698 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:12.732974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:12.733241 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] PATH_INFO: `/v3/users/d4e9b26a95ec451eaed6c41385f44957` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:12.734379 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e9b26a95ec451eaed6c41385f44957)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:12.736570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:12.795210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e9b26a95ec451eaed6c41385f44957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.804337 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e9b26a95ec451eaed6c41385f44957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.812870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e9b26a95ec451eaed6c41385f44957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.813676 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e9b26a95ec451eaed6c41385f44957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.827945 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e9b26a95ec451eaed6c41385f44957'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:12.834231 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1438/2872] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:12 2022] DELETE /identity/v3/users/d4e9b26a95ec451eaed6c41385f44957 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:13.019297 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:13.020728 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:13.033734 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e2dde-c49d-46b9-8f0b-dc86f0a68bea admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:13.090168 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:13.090827 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:13.091101 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:13.091403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] PATH_INFO: `/v3/projects/1b367e6b2c73466cbb5b25183fbee53a` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:13.092543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b367e6b2c73466cbb5b25183fbee53a)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:13.094424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:13.147974 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b367e6b2c73466cbb5b25183fbee53a', 'request_id': 'req-233f039c-ee2a-4963-ad9a-ebd918d8811e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:13.156573 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b367e6b2c73466cbb5b25183fbee53a', 'request_id': 'req-233f039c-ee2a-4963-ad9a-ebd918d8811e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:13.158574 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:13.163081 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b367e6b2c73466cbb5b25183fbee53a', 'request_id': 'req-233f039c-ee2a-4963-ad9a-ebd918d8811e'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:13.175505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] The token cache is being invalidate because project 1b367e6b2c73466cbb5b25183fbee53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:13.175780 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:13.176409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-233f039c-ee2a-4963-ad9a-ebd918d8811e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:13.177922 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1435/2873] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:13 2022] DELETE /identity/v3/projects/1b367e6b2c73466cbb5b25183fbee53a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:22.830850 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:23.172459 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=5,keystone:DELETE=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:25.223733 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:25.225365 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:25.239001 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8984d8a2-8b95-4097-ac1b-edb8a0c69db6 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:25.299486 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:25.300119 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:25.300388 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:25.300650 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] PATH_INFO: `/v3/users/c9cff541ea324815b62b8710e7ce9dc6` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:25.301806 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9cff541ea324815b62b8710e7ce9dc6)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:25.303736 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:25.354105 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9cff541ea324815b62b8710e7ce9dc6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.362887 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9cff541ea324815b62b8710e7ce9dc6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.371476 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9cff541ea324815b62b8710e7ce9dc6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.372178 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9cff541ea324815b62b8710e7ce9dc6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.384599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9cff541ea324815b62b8710e7ce9dc6'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.390518 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1439/2874] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:25 2022] DELETE /identity/v3/users/c9cff541ea324815b62b8710e7ce9dc6 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:25.395085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:25.395632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:25.395632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:25.414898 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:25.416109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:25.478684 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1436/2875] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:25.489353 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:25.490831 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:25.553358 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1ecf8-e334-4bf2-9a56-fd64f53ab679 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:25.557597 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:25.558206 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:25.558480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:25.558747 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:25.568568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:25.570104 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:25.580078 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1440/2876] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:25.855348 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:25.856887 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:25.869456 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b788d-12c9-4742-8504-5468fbb1029d None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:25.875754 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:25.876504 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:25.876836 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:25.877229 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] PATH_INFO: `/v3/projects/2dfc3d000db842ea8dafb4fb69a4166c` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:25.878428 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfc3d000db842ea8dafb4fb69a4166c)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:25.880583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:25.930129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfc3d000db842ea8dafb4fb69a4166c', 'request_id': 'req-24979aeb-7897-4760-8257-cb18b0ed4bb3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.938206 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfc3d000db842ea8dafb4fb69a4166c', 'request_id': 'req-24979aeb-7897-4760-8257-cb18b0ed4bb3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.943895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfc3d000db842ea8dafb4fb69a4166c', 'request_id': 'req-24979aeb-7897-4760-8257-cb18b0ed4bb3'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.954632 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] The token cache is being invalidate because project 2dfc3d000db842ea8dafb4fb69a4166c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:25.954918 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.955560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:25.957055 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1437/2877] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:25 2022] DELETE /identity/v3/projects/2dfc3d000db842ea8dafb4fb69a4166c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:25.961313 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:25.971325 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f3f5-645e-4eaa-988a-e8b9a3c95869 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:25.990012 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:26.031755 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.032421 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.032695 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.033056 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] PATH_INFO: `/v3/users/1f3c441381c845f9bd1e6a1587d41e26` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.034084 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f3c441381c845f9bd1e6a1587d41e26)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.036396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.086411 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f3c441381c845f9bd1e6a1587d41e26'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.095523 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f3c441381c845f9bd1e6a1587d41e26'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.102599 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f3c441381c845f9bd1e6a1587d41e26'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.103277 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f3c441381c845f9bd1e6a1587d41e26'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.115524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-95ac1e63-574a-40a9-9965-83ef20129a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f3c441381c845f9bd1e6a1587d41e26'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.121865 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1441/2878] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:25 2022] DELETE /identity/v3/users/1f3c441381c845f9bd1e6a1587d41e26 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:26.400332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.401909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:26.416463 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24979aeb-7897-4760-8257-cb18b0ed4bb3 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.475821 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.476438 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.476701 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.476940 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] PATH_INFO: `/v3/projects/8edc5007b81a49658f8b1a16c2fd8ab8` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.478116 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edc5007b81a49658f8b1a16c2fd8ab8)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.480050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.511141 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:26.529561 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edc5007b81a49658f8b1a16c2fd8ab8', 'request_id': 'req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.538102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edc5007b81a49658f8b1a16c2fd8ab8', 'request_id': 'req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.544380 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edc5007b81a49658f8b1a16c2fd8ab8', 'request_id': 'req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.556026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] The token cache is being invalidate because project 8edc5007b81a49658f8b1a16c2fd8ab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:26.556306 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.557007 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-3a49591b-1a29-4c8f-bfc4-de5bcd749c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:26.558610 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1438/2879] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:26 2022] DELETE /identity/v3/projects/8edc5007b81a49658f8b1a16c2fd8ab8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:26.563866 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.564387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.564498 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.585898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:26.587113 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:26.653674 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1442/2880] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:26.664779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.665131 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.665490 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.690491 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:26.691770 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:26.702268 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1439/2881] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:26.706655 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.717252 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2f93b-c75e-4318-a713-fe50e37e4b46 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.722977 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.723604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.723874 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.724132 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.724773 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81487) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:03:26.725088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.726738 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.740144 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1443/2882] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:03:26.744604 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.754788 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa9ef3a-0daf-4851-8b37-597600bbe0f6 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.760776 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.761366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.761630 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.761862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.762505 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.764426 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.797934 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1440/2883] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:26 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:26.800280 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.813217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ddb9a-421b-4ac2-a94f-c97f68c804eb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.839934 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:26.875300 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.875913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.876186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.876451 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.877082 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.878988 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.902224 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1444/2884] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:26 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:26.906596 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.917747 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b4fc3-16ee-4d04-a051-33f53b901cb6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.924326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.925197 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.925522 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.925803 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.926790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.929031 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.939826 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1441/2885] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:26.944077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.955305 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d44edb-c6f7-491a-89e5-b680dde0a3cb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.960938 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdf8bd-1395-4c4c-92d3-6183386ba9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.961555 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdf8bd-1395-4c4c-92d3-6183386ba9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.961854 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdf8bd-1395-4c4c-92d3-6183386ba9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.962049 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdf8bd-1395-4c4c-92d3-6183386ba9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:26.962711 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdf8bd-1395-4c4c-92d3-6183386ba9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:26.964307 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdf8bd-1395-4c4c-92d3-6183386ba9b0 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:26.974911 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1445/2886] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:26.979338 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:26.989559 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eab99-9a5d-4ab6-98e8-8a28333869f6 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:26.994725 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:26.995554 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:26.995945 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:26.996332 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] PATH_INFO: `/v3/projects/c11f2e0d37fb4a29ab076421e48e529c/users/d8a4c65c1b184946bc4d7354413f0f2f/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.012289 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11f2e0d37fb4a29ab076421e48e529c, user_id=d8a4c65c1b184946bc4d7354413f0f2f, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.014885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.029496 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1442/2887] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:26 2022] PUT /identity/v3/projects/c11f2e0d37fb4a29ab076421e48e529c/users/d8a4c65c1b184946bc4d7354413f0f2f/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:27.041673 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.041948 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.042215 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.071971 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] MFA Rules not processed for user `d8a4c65c1b184946bc4d7354413f0f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:27.073179 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:27.142593 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1446/2888] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:27.147469 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.232417 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428c21a-7778-42f2-accf-6097421db8bf admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.238547 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.239158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.239460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.239737 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] PATH_INFO: `/v3/projects` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.240359 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.242304 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.271905 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1443/2889] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:27 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:27.276559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.288263 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d83993-e0a0-4be7-b150-474cf5b1dce9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.346099 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.346697 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.347000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.347288 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] PATH_INFO: `/v3/users` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.347908 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.349826 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.374239 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1447/2890] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:27 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:27.379065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.389998 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d3178-6189-4029-b1f7-248ea0d78153 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.395746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.396344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.396634 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.396895 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.397565 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.399314 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.413493 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1444/2891] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:27.419042 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.429846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636509c-3d22-494c-9cc0-5cd363df4502 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.434957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.435631 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.435910 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.436180 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] PATH_INFO: `/v3/projects/8d7d57f411214115aaf335ca943db76f/users/ae4a4961afed4baf81fb54435885bcd5/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.455888 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7d57f411214115aaf335ca943db76f, user_id=ae4a4961afed4baf81fb54435885bcd5, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.458528 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.481313 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1448/2892] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:27 2022] PUT /identity/v3/projects/8d7d57f411214115aaf335ca943db76f/users/ae4a4961afed4baf81fb54435885bcd5/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:27.486270 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.498686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7986f02c-1ed4-478a-9cc5-eb23ee42515f admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.518377 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:27.557643 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4362f4f-79cc-42f0-8ead-ae6c22064d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.558239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4362f4f-79cc-42f0-8ead-ae6c22064d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.558513 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4362f4f-79cc-42f0-8ead-ae6c22064d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.558777 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4362f4f-79cc-42f0-8ead-ae6c22064d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.559454 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4362f4f-79cc-42f0-8ead-ae6c22064d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.561050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4362f4f-79cc-42f0-8ead-ae6c22064d85 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.570822 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1445/2893] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:27.576020 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.583723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:27.591514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c173f-6df1-41bf-ba96-b0df6046b980 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.597865 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.598527 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.598801 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.599073 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] PATH_INFO: `/v3/projects/8d7d57f411214115aaf335ca943db76f/users/ae4a4961afed4baf81fb54435885bcd5/roles/77132358e0b64de4a63750b81e359340` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.601166 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d7d57f411214115aaf335ca943db76f, user_id=ae4a4961afed4baf81fb54435885bcd5, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.603870 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.615095 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1449/2894] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:27 2022] PUT /identity/v3/projects/8d7d57f411214115aaf335ca943db76f/users/ae4a4961afed4baf81fb54435885bcd5/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:27.625552 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.625831 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.626102 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.654267 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] MFA Rules not processed for user `ae4a4961afed4baf81fb54435885bcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:27.655508 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:27.733860 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1446/2895] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:27.745432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:27.747409 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:27.814319 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab2ef9-50a5-4f2d-aedc-ca81a0470172 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:27.820590 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db777f-0247-4198-848e-16886d24b999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:27.821187 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db777f-0247-4198-848e-16886d24b999 service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:27.821456 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db777f-0247-4198-848e-16886d24b999 service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:27.821710 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db777f-0247-4198-848e-16886d24b999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:27.831930 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db777f-0247-4198-848e-16886d24b999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:27.833355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db777f-0247-4198-848e-16886d24b999 service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:27.842830 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1450/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:27.980992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:28.055492 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b867fd4c-31d0-4b00-b571-97c5eae02929 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:28.060663 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:28.061287 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:28.061555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:28.061814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] PATH_INFO: `/v3/users/d8a4c65c1b184946bc4d7354413f0f2f` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:28.063015 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a4c65c1b184946bc4d7354413f0f2f)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:28.064988 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:28.117257 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a4c65c1b184946bc4d7354413f0f2f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.118979 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:28.125707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a4c65c1b184946bc4d7354413f0f2f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.137930 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a4c65c1b184946bc4d7354413f0f2f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.138571 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a4c65c1b184946bc4d7354413f0f2f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.146068 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a4c65c1b184946bc4d7354413f0f2f'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.152808 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1447/2897] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:27 2022] DELETE /identity/v3/users/d8a4c65c1b184946bc4d7354413f0f2f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:28.157669 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:28.158000 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:28.158351 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:28.178661 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:28.179968 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:28.240385 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1451/2898] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:28.250473 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:28.252038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:28.309909 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b632372-b491-43fb-a27a-c55589e6ed9e admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:28.313686 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:28.314302 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:28.314596 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:28.314856 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:28.324796 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:28.326185 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:28.335689 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1448/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:28.622091 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:28.623723 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:28.636716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7184a-e2f5-4bce-9eda-834a6b301846 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:28.641943 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:28.642566 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:28.642863 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:28.643154 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] PATH_INFO: `/v3/projects/c11f2e0d37fb4a29ab076421e48e529c` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:28.644481 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11f2e0d37fb4a29ab076421e48e529c)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:28.646534 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:28.699491 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11f2e0d37fb4a29ab076421e48e529c', 'request_id': 'req-246b71dd-c153-424c-bb63-1b0798c65b80'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.707824 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11f2e0d37fb4a29ab076421e48e529c', 'request_id': 'req-246b71dd-c153-424c-bb63-1b0798c65b80'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.714557 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11f2e0d37fb4a29ab076421e48e529c', 'request_id': 'req-246b71dd-c153-424c-bb63-1b0798c65b80'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.726248 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] The token cache is being invalidate because project c11f2e0d37fb4a29ab076421e48e529c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:28.726539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.727320 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.728891 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1452/2900] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:28 2022] DELETE /identity/v3/projects/c11f2e0d37fb4a29ab076421e48e529c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:28.733103 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:28.742996 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39b81-691d-4b1b-8516-ebc0490033d7 service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:28.804489 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:28.805137 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:28.805409 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:28.805699 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] PATH_INFO: `/v3/users/ae4a4961afed4baf81fb54435885bcd5` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:28.806862 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae4a4961afed4baf81fb54435885bcd5)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:28.808805 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:28.855414 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae4a4961afed4baf81fb54435885bcd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.864085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae4a4961afed4baf81fb54435885bcd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.875449 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:28.877190 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae4a4961afed4baf81fb54435885bcd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.877948 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae4a4961afed4baf81fb54435885bcd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.886555 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-350c0f77-77ba-415b-b352-f17bf863a4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae4a4961afed4baf81fb54435885bcd5'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:28.892470 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1449/2901] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:28 2022] DELETE /identity/v3/users/ae4a4961afed4baf81fb54435885bcd5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:29.181974 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.183495 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:29.196829 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b71dd-c153-424c-bb63-1b0798c65b80 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.209835 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:29.256267 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.256903 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.257201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.257568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] PATH_INFO: `/v3/projects/8d7d57f411214115aaf335ca943db76f` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.258688 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d7d57f411214115aaf335ca943db76f)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.260630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.310657 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d7d57f411214115aaf335ca943db76f', 'request_id': 'req-431de35a-7631-4b79-82b2-376e17c1ae7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:29.320102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d7d57f411214115aaf335ca943db76f', 'request_id': 'req-431de35a-7631-4b79-82b2-376e17c1ae7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:29.327221 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d7d57f411214115aaf335ca943db76f', 'request_id': 'req-431de35a-7631-4b79-82b2-376e17c1ae7c'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:29.338904 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] The token cache is being invalidate because project 8d7d57f411214115aaf335ca943db76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:29.339214 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:29.340013 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-431de35a-7631-4b79-82b2-376e17c1ae7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:29.341640 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1453/2902] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:29 2022] DELETE /identity/v3/projects/8d7d57f411214115aaf335ca943db76f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:29.346697 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.347129 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.347425 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.368241 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:29.369403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:29.435405 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1450/2903] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:29.447276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.447554 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.447823 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.468229 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:29.469430 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:29.477771 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1454/2904] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:29.482238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.492222 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689b46e-31d8-4088-81bc-c349aa595b7b None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.496822 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.497408 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.497707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.498142 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.498652 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:03:29.498952 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.500715 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.514843 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1451/2905] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:03:29.519316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.528730 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d63dd4-6e03-401e-964b-56042645c450 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.534490 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.535092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.535398 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.536023 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.536662 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.538648 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.574281 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1455/2906] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:29 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:29.578616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.589424 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab80-1e6f-4b2e-a130-ad4e50ca378a admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.656026 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.656534 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.656793 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.657057 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.657689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.659653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.684026 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1452/2907] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:29 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:29.688391 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.699269 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03134c-c0f4-4383-9dfb-8c8f0e7599ab admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.704238 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.704906 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.705274 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.705546 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.706192 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.707814 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.718326 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1456/2908] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:03:29.722576 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.733228 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f813a-d089-4b55-b83a-8d994a3aa439 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.738779 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486532b-28a9-4e3c-a01f-44dd71c5cecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.739403 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486532b-28a9-4e3c-a01f-44dd71c5cecf admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.739672 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486532b-28a9-4e3c-a01f-44dd71c5cecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.739928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486532b-28a9-4e3c-a01f-44dd71c5cecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.740560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486532b-28a9-4e3c-a01f-44dd71c5cecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.742319 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486532b-28a9-4e3c-a01f-44dd71c5cecf admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.752900 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1453/2909] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:29.757005 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.767647 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b0d9b-98ea-4e4b-bff6-ac7805752c5a admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:29.773075 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:29.773754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.774044 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.774309 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] PATH_INFO: `/v3/projects/2adc7180b8da4b6e9826de58dc32dcc5/users/874884e831ca426f8282e67f636406ca/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.790429 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adc7180b8da4b6e9826de58dc32dcc5, user_id=874884e831ca426f8282e67f636406ca, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:29.793064 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:29.809432 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1457/2910] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:29 2022] PUT /identity/v3/projects/2adc7180b8da4b6e9826de58dc32dcc5/users/874884e831ca426f8282e67f636406ca/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:29.879316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68e99-2759-4887-8478-e72725259749 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:29.879560 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68e99-2759-4887-8478-e72725259749 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:29.879849 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a68e99-2759-4887-8478-e72725259749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:29.909045 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-55a68e99-2759-4887-8478-e72725259749 None None] MFA Rules not processed for user `874884e831ca426f8282e67f636406ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:29.910256 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-55a68e99-2759-4887-8478-e72725259749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:29.953529 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:29.984246 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1454/2911] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:29.997156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:29.999077 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:30.047925 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:30.061477 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d6b24-2c67-4692-9f5c-01b5049f7e74 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:30.065642 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb312c4-e483-4438-8de1-342b3a18387c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d71b946e7d64a50888cfcc337d688a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:30.066245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb312c4-e483-4438-8de1-342b3a18387c service cinder] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:30.066519 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb312c4-e483-4438-8de1-342b3a18387c service cinder] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:30.066793 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb312c4-e483-4438-8de1-342b3a18387c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:30.075473 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb312c4-e483-4438-8de1-342b3a18387c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:30.076899 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb312c4-e483-4438-8de1-342b3a18387c service cinder] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:30.085754 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1458/2912] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:33.188315 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68e99-2759-4887-8478-e72725259749 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:33.256892 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a68e99-2759-4887-8478-e72725259749 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:33.261657 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:33.262279 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:33.262548 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:33.262811 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] PATH_INFO: `/v3/users/874884e831ca426f8282e67f636406ca` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:33.263965 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874884e831ca426f8282e67f636406ca)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:33.266047 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:33.314195 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874884e831ca426f8282e67f636406ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.322094 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874884e831ca426f8282e67f636406ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.333994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874884e831ca426f8282e67f636406ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.334628 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874884e831ca426f8282e67f636406ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.341665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874884e831ca426f8282e67f636406ca'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.348010 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1455/2913] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:03:33 2022] DELETE /identity/v3/users/874884e831ca426f8282e67f636406ca => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:33.352644 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:33.353116 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:33.353217 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:33.374086 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:33.375334 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:33.442261 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1459/2914] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:33.452218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:33.453732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:33.515619 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc8ff2-ec81-4b7e-b833-df57d3369b58 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:33.519933 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd21ba-3a08-4999-8c0b-d458c0173976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:33.520533 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd21ba-3a08-4999-8c0b-d458c0173976 service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:33.520806 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd21ba-3a08-4999-8c0b-d458c0173976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:33.521069 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd21ba-3a08-4999-8c0b-d458c0173976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:33.531278 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd21ba-3a08-4999-8c0b-d458c0173976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:33.532670 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd21ba-3a08-4999-8c0b-d458c0173976 service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:33.542935 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1456/2915] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:33.815994 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:33.817472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:33.829913 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8c7f2-7a52-472a-990c-4451d66692fc None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:33.835559 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:33.836199 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:33.836472 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:33.836898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] PATH_INFO: `/v3/projects/2adc7180b8da4b6e9826de58dc32dcc5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:33.838131 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adc7180b8da4b6e9826de58dc32dcc5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:33.840112 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:33.890396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adc7180b8da4b6e9826de58dc32dcc5', 'request_id': 'req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.897699 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adc7180b8da4b6e9826de58dc32dcc5', 'request_id': 'req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.903921 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adc7180b8da4b6e9826de58dc32dcc5', 'request_id': 'req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.915134 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] The token cache is being invalidate because project 2adc7180b8da4b6e9826de58dc32dcc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81487) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:03:33.915445 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.916286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-00b6fe76-5f8b-49e6-a7e8-ab9a088f0132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:03:33.917916 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1460/2916] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:03:33 2022] DELETE /identity/v3/projects/2adc7180b8da4b6e9826de58dc32dcc5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:33.924665 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:33.924944 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:33.925239 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:33.934232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:33.947008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:33.948735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:34.008817 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1457/2917] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:34.016716 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.017088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.017381 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.037685 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:34.038889 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:34.047792 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1461/2918] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:34.052350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.063333 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabfca6-5ed7-4bf5-9eb0-be5a9cff4219 None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.069082 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.069683 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.069953 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.070213 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.070885 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81488) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 11 01:03:34.071238 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.072834 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.084936 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1458/2919] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Sep 11 01:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 11 01:03:34.089784 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.099432 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd797fd-a3bf-469f-b77d-a5b5478a7ca9 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.104499 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.105102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.105373 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.105630 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.106271 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.108203 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.137210 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1462/2920] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:34 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:34.141557 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.151900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb07162-f0fa-472d-b8df-4ab7d59682d9 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.209109 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.209801 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.209923 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.210158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.210789 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.212735 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.243133 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1459/2921] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:34 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:34.247861 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.259089 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ccd378-ab7f-41e8-acc2-a2a004733295 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.264727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.265327 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.265604 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.265873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.266520 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.268169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.278825 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1463/2922] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:34.283028 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.298983 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d682b41-8f60-4142-8ab2-42b7189e8393 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.307721 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8ca36-8d9d-48eb-9bf7-16ab7128139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.308316 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ca36-8d9d-48eb-9bf7-16ab7128139c admin admin] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.308583 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ca36-8d9d-48eb-9bf7-16ab7128139c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.308845 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8ca36-8d9d-48eb-9bf7-16ab7128139c admin admin] PATH_INFO: `/v3/roles` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.309481 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ca36-8d9d-48eb-9bf7-16ab7128139c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.311065 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8ca36-8d9d-48eb-9bf7-16ab7128139c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.322121 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1460/2923] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:34.326401 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.336797 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a35efca-2642-442f-bdfc-1b2db305f6fb admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.341933 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.342570 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.342838 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.343101 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] PATH_INFO: `/v3/projects/e7b47e4ca8c0414790b8d0fa30559d25/users/c97d41a8f1ff40e79472aa4bf9790ae5/roles/1610cc07f3b24392a709fa07d663cb77` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.358125 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7b47e4ca8c0414790b8d0fa30559d25, user_id=c97d41a8f1ff40e79472aa4bf9790ae5, role_id=1610cc07f3b24392a709fa07d663cb77)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.360857 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.375362 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1464/2924] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:34 2022] PUT /identity/v3/projects/e7b47e4ca8c0414790b8d0fa30559d25/users/c97d41a8f1ff40e79472aa4bf9790ae5/roles/1610cc07f3b24392a709fa07d663cb77 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:34.380328 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100db3e-747e-429c-b3e1-93041862e82c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.380610 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100db3e-747e-429c-b3e1-93041862e82c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.380872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100db3e-747e-429c-b3e1-93041862e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.399911 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-d100db3e-747e-429c-b3e1-93041862e82c None None] MFA Rules not processed for user `cf7ec619bcef4004bcdaf4a26a07260b`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:34.401232 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-d100db3e-747e-429c-b3e1-93041862e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:34.463713 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1461/2925] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:34.476173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:34.477972 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:34.485279 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:34.538727 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8f358-f3fd-48cb-a84c-0da51253fb5c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:34.542640 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:34.543239 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:34.543596 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:34.543849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:34.553193 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:34.554613 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:34.564127 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1465/2926] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:38.548395 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:38.548976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:38.548976 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:38.567170 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:38.592323 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.auth.core [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] MFA Rules not processed for user `c97d41a8f1ff40e79472aa4bf9790ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81488) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:38.593567 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:38.692673 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1462/2927] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:38.697651 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:38.759396 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e493c-566e-43e6-a9c8-28e6c3d0d94a service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:38.764496 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:38.765220 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:38.765393 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:38.766315 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] PATH_INFO: `/v3/projects` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:38.767138 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:38.771268 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:38.818502 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1466/2928] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Sep 11 01:03:38 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:38.819833 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:38.832225 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d6dc8-753b-407e-9538-b4f6c846cb6c None None] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:38.897008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:38.897543 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] REQUEST_METHOD: `POST` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:38.897814 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:38.898080 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] PATH_INFO: `/v3/users` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:38.898712 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:38.900746 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:38.924629 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1463/2929] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Sep 11 01:03:38 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 11 01:03:38.928533 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:38.939384 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2e0cd-df5c-4071-a6de-dd92b6d7f341 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:38.945198 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:38.945912 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:38.946186 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:38.946898 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:38.947218 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:38.948846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:38.961224 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1467/2930] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 11 01:03:38.965173 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:38.976008 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d09d3d2-d596-495e-8939-4e78bee1bc92 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:38.981114 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:38.981815 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:38.982049 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:38.982326 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] PATH_INFO: `/v3/projects/4c7aba9ccf094d928c0a45ec6d3649b7/users/ff905533e9224ffc81388cfe36e0c319/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:38.999078 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7aba9ccf094d928c0a45ec6d3649b7, user_id=ff905533e9224ffc81388cfe36e0c319, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:39.001994 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:39.017332 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1464/2931] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:38 2022] PUT /identity/v3/projects/4c7aba9ccf094d928c0a45ec6d3649b7/users/ff905533e9224ffc81388cfe36e0c319/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:03:39.021514 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:39.031539 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f09f7-0aa2-4cea-b03c-fb9c2a6dd1a5 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:39.097455 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917e2f2-4e95-4f05-a273-413ef79b42d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:39.098173 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e2f2-4e95-4f05-a273-413ef79b42d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:39.098515 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e2f2-4e95-4f05-a273-413ef79b42d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:39.098849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917e2f2-4e95-4f05-a273-413ef79b42d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:39.100513 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e2f2-4e95-4f05-a273-413ef79b42d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:39.102196 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917e2f2-4e95-4f05-a273-413ef79b42d4 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:39.114786 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1468/2932] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Sep 11 01:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 11 01:03:39.119457 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:39.130772 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0923c6-34d9-4036-a757-020afff5a886 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:39.137268 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:39.137949 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:39.138217 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:39.138477 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] PATH_INFO: `/v3/projects/4c7aba9ccf094d928c0a45ec6d3649b7/users/ff905533e9224ffc81388cfe36e0c319/roles/77132358e0b64de4a63750b81e359340` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:39.141593 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7aba9ccf094d928c0a45ec6d3649b7, user_id=ff905533e9224ffc81388cfe36e0c319, role_id=77132358e0b64de4a63750b81e359340)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:39.145364 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:39.157319 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1465/2933] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Sep 11 01:03:39 2022] PUT /identity/v3/projects/4c7aba9ccf094d928c0a45ec6d3649b7/users/ff905533e9224ffc81388cfe36e0c319/roles/77132358e0b64de4a63750b81e359340 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:03:43.526275 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014635d-0fd9-4deb-b870-dbda203454ef None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:43.526825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014635d-0fd9-4deb-b870-dbda203454ef None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:43.526825 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014635d-0fd9-4deb-b870-dbda203454ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:43.580355 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-6014635d-0fd9-4deb-b870-dbda203454ef None None] MFA Rules not processed for user `ff905533e9224ffc81388cfe36e0c319`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:43.581846 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-6014635d-0fd9-4deb-b870-dbda203454ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:43.590947 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:43.663482 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1469/2934] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:43.690704 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:43.695283 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:43.742158 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:43.843698 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702e8c-cdf5-46b0-bd44-52628a4509e4 admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:43.849689 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:43.850305 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:43.850575 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:43.850835 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:43.909112 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:43.911022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:43.920732 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1466/2935] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:03:45.212088 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] REQUEST_METHOD: `POST` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:45.212654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:45.212654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:45.234124 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.auth.core [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] MFA Rules not processed for user `c97d41a8f1ff40e79472aa4bf9790ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81487) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 11 01:03:45.235602 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:45.245174 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1470/2936] 10.222.0.23 () {58 vars in 958 bytes} [Sun Sep 11 01:03:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 11 01:03:53.916350 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=63 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:55.228525 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=28 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:03:59.702868 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:03:59.705085 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:59.796243 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315df146-48a6-4cb5-b2e1-fdd6fb89ee66 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:03:59.801872 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:03:59.802480 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:03:59.802753 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:03:59.803022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:03:59.805040 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:03:59.817125 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:03:59.818928 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:03:59.829117 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1467/2937] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:03:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 11 01:04:09.824616 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=41 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:04:11.307524 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:04:11.309369 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:04:11.395149 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cdaf1a-ede2-46c8-b955-47d1e81fe4f2 None None] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:04:11.399107 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:04:11.399725 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:04:11.399997 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:04:11.400253 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:04:11.410156 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:04:11.411606 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:04:11.421847 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1471/2938] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:04:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:04:21.417537 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=35 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:06:07.364915 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:06:07.377022 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15308729-62c0-4437-bf82-b91a370ea592 service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:06:07.381166 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:06:07.381782 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:06:07.382050 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:06:07.382312 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:06:07.384366 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:06:07.398797 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:06:07.400245 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:06:07.411731 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1468/2939] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:06:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:06:08.598387 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:06:08.607439 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9df169-77dd-42f0-8367-1a284505f36a service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:06:08.611634 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:06:08.612244 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:06:08.612480 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:06:08.612754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:06:08.614931 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:06:08.634286 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:06:08.635743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:06:08.647159 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1472/2940] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:06:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:06:17.406545 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:06:17.600345 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:06:17.609783 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60915ee6-5ec7-4ea4-87d3-36bc1acd85bf service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:06:17.614394 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '341595ae0d9b4431b4417edda29f03ff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:06:17.614992 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:06:17.615294 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:06:17.615592 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:06:17.617680 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:06:17.628662 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:06:17.630118 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:06:17.639098 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1469/2941] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:06:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:06:18.642210 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:06:20.961820 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:06:20.963418 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:06:20.977416 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b45048-5207-48ac-aece-5b9cabad42f5 service neutron] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:06:20.982568 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:06:20.983201 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:06:20.983475 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:06:20.983743 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:06:20.985565 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:06:20.997282 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:06:20.998841 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:06:21.008949 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1473/2942] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:06:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:06:27.635087 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=6 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:06:31.004848 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=6 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:07:36.370808 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:36.372883 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:07:36.393941 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83356f10-1386-4503-b9a8-e270e9c94bfd service neutron] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:36.400415 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:36.401038 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:36.401292 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:36.401584 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:36.403479 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:07:36.417732 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:36.419352 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:36.430980 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1470/2943] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:07:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:07:39.549508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:39.565162 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8d2a2-c09d-41bc-9b59-e46a3ed0d09b service placement] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:39.570798 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c150143af9144547ab3dc5a5d318d45f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:39.571917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:39.571917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:39.571917 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:39.573939 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:07:39.588654 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:39.590245 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:39.601549 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1474/2944] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:07:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:07:44.387385 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:44.400189 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c5b87-0473-4c45-9a78-83ec4d319250 service nova] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:44.406959 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:44.407775 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] REQUEST_METHOD: `GET` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:44.408156 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:44.408618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:44.410651 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:07:44.425879 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:44.427738 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:44.449285 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1471/2945] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:07:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:07:47.731298 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:47.810529 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f2995-423b-4f94-84f7-b96aa0e72510 service nova] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:47.815709 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:47.816338 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:47.816610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:47.816873 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] PATH_INFO: `/v3/users/c97d41a8f1ff40e79472aa4bf9790ae5` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:47.818008 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c97d41a8f1ff40e79472aa4bf9790ae5)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:47.819957 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:47.870332 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c97d41a8f1ff40e79472aa4bf9790ae5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:47.881406 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c97d41a8f1ff40e79472aa4bf9790ae5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:47.888468 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c97d41a8f1ff40e79472aa4bf9790ae5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:47.889102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c97d41a8f1ff40e79472aa4bf9790ae5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:47.900978 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c97d41a8f1ff40e79472aa4bf9790ae5'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:47.907369 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1475/2946] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:07:47 2022] DELETE /identity/v3/users/c97d41a8f1ff40e79472aa4bf9790ae5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:07:48.081320 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:48.093606 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42faab8-937a-4d99-9a26-c9f221d411be service placement] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:48.154467 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:48.155107 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:48.155516 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:48.155790 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] PATH_INFO: `/v3/projects/e7b47e4ca8c0414790b8d0fa30559d25` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:48.157303 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7b47e4ca8c0414790b8d0fa30559d25)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:48.159707 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:48.209799 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7b47e4ca8c0414790b8d0fa30559d25', 'request_id': 'req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.218344 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7b47e4ca8c0414790b8d0fa30559d25', 'request_id': 'req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.224399 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7b47e4ca8c0414790b8d0fa30559d25', 'request_id': 'req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.236375 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] The token cache is being invalidate because project e7b47e4ca8c0414790b8d0fa30559d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:07:48.236653 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.237272 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.238743 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1472/2947] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:07:48 2022] DELETE /identity/v3/projects/e7b47e4ca8c0414790b8d0fa30559d25 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 11 01:07:48.244169 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:48.246294 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:07:48.265677 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0affb7-f464-4bdb-892d-11854b754781 admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:48.329610 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:48.330234 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:48.330508 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:48.330772 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] PATH_INFO: `/v3/users/ff905533e9224ffc81388cfe36e0c319` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:48.331985 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff905533e9224ffc81388cfe36e0c319)` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:48.333881 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:48.339885 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:07:48.383718 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff905533e9224ffc81388cfe36e0c319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.391447 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff905533e9224ffc81388cfe36e0c319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.398466 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff905533e9224ffc81388cfe36e0c319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.399092 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff905533e9224ffc81388cfe36e0c319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.411316 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.notifications [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff905533e9224ffc81388cfe36e0c319'} {{(pid=81487) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.417138 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1476/2948] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Sep 11 01:07:48 2022] DELETE /identity/v3/users/ff905533e9224ffc81388cfe36e0c319 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:07:48.588631 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Authenticating user token {{(pid=81488) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:07:48.590420 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.fernet_utils [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81488) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:07:48.606927 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4805b2-cf0f-44e1-87cf-1b3b66deb77c admin admin] Validating token access rules against request {{(pid=81488) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:07:48.666271 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf7ec619bcef4004bcdaf4a26a07260b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '857538f273814e30bf74c0248d309754', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81488) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:07:48.666900 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:07:48.667171 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:07:48.667460 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] PATH_INFO: `/v3/projects/4c7aba9ccf094d928c0a45ec6d3649b7` {{(pid=81488) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:07:48.668550 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7aba9ccf094d928c0a45ec6d3649b7)` {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:07:48.670431 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] RBAC: Authorization granted {{(pid=81488) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:07:48.676618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:07:48.718989 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7aba9ccf094d928c0a45ec6d3649b7', 'request_id': 'req-93c2a7a6-a000-478a-9420-a063702c3316'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.727618 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7aba9ccf094d928c0a45ec6d3649b7', 'request_id': 'req-93c2a7a6-a000-478a-9420-a063702c3316'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.734029 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7aba9ccf094d928c0a45ec6d3649b7', 'request_id': 'req-93c2a7a6-a000-478a-9420-a063702c3316'} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.745218 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] The token cache is being invalidate because project 4c7aba9ccf094d928c0a45ec6d3649b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81488) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 11 01:07:48.745500 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.746231 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG keystone.notifications [None req-93c2a7a6-a000-478a-9420-a063702c3316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81488) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 11 01:07:48.747930 dv-cinder-master-3-smb devstack@keystone.service[81488]: [pid: 81488|app: 0|req: 1473/2949] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Sep 11 01:07:48 2022] DELETE /identity/v3/projects/4c7aba9ccf094d928c0a45ec6d3649b7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 11 01:07:58.414019 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:07:58.742108 dv-cinder-master-3-smb devstack@keystone.service[81488]: DEBUG dbcounter [-] [81488] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81488) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 11 01:09:49.763811 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Authenticating user token {{(pid=81487) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 11 01:09:49.765843 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:09:49.856284 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0247d8-4dff-431b-a892-5038a83af70c admin admin] Validating token access rules against request {{(pid=81487) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 11 01:09:49.927302 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00e3a0e394ba43f3a2362c3ac69a2fa6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'be8a77b9c6d04dfb826b0a857fcb6087', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81487) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 11 01:09:49.927849 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] REQUEST_METHOD: `GET` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 11 01:09:49.928102 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] SCRIPT_NAME: `/identity` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 11 01:09:49.928392 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81487) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 11 01:09:49.930276 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.fernet_utils [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81487) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 11 01:09:49.942152 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 11 01:09:49.943754 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc351e-cef1-4af8-b90b-e952c9c41983 service placement] RBAC: Authorization granted {{(pid=81487) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 11 01:09:49.954405 dv-cinder-master-3-smb devstack@keystone.service[81487]: [pid: 81487|app: 0|req: 1477/2950] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Sep 11 01:09:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 11 01:09:59.949705 dv-cinder-master-3-smb devstack@keystone.service[81487]: DEBUG dbcounter [-] [81487] Writing DB stats keystone:SELECT=58 {{(pid=81487) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}